NVIDIA Next? Predicting Peaks and Spotting Sneaks in Stock Prices

Introduction

What is a Time Series??⏲📈

Time-series is series of obeservations that are recorded over a period of time. these observations are dependent of time component which can not be neglected thus we have have to analysis the this data keeping time component in mind.

Time Series Forecasting

Time series forecasting is parhaps one of the most common type of machine learning techniques used in real-world scenarios. time-sereis forecsting refers to predicting future values from historical data by statical analysis of trends and patterns from certain time-series data. it falls under unsupervised learning category but called as a self-supervised learning or supervised learning technique. time-series data can be much complex to find patterns out of it, this is because irregular component of time series. </div>

Motivation

The motivation behind this project is to use deep learning for predicting the stock prices of Nvidia, which can be a complex and dynamic task. The goal is to develop supervised learning algorithms using Long Short-Term Memory networks (LSTMs) for time series forecasting. LSTM networks are particularly well-suited for financial time series data because they can capture long-term dependencies and patterns that are crucial for accurate predictions.

In conclusion, this paper presents a dataset of Nvidia’s stock prices and demonstrates the effectiveness of LSTM models in predicting financial markets. The dataset and the models can serve as a benchmark for developing more sophisticated financial forecasting systems, which can potentially help investors and financial analysts in making informed decisions.

Dataset Source

The ”Nvidia Stock Price Dataset” For the experimental study, I downloaded live dataset, namely Nvidia, from the Yahoo Finance website Yahoo Finance. The dataset includes historical stock price data for Nvidia over the past 10 years, enriched with several technical indicators to analyze market trends and stock performance. It details the date, as well as the opening, highest, lowest, and closing prices for each trading day, and also the volume of shares traded.

The Nvidia stock price dataset comprises 2518 samples. The data is first pre-processed to handle any missing values, normalize figures, and derive necessary technical indicators before being split into training and testing sets. Typically, 80% of the data is used for training the LSTM model, while the remaining 20% is reserved for validating/testing its performance. This split helps in evaluating the model’s effectiveness in predicting stock movements under varied market conditions. The input features pertain to certain fields: Open, High, Low, Close, Adj Close, Volume

Loading Python Libraries

# Let's import the required python packages for next day rain predict
import numpy as np
import pandas as pd
import seaborn as sns
import yfinance as yf
import tensorflow as tf
import missingno as msno
import plotly.graph_objects as go
from sklearn.preprocessing import MinMaxScaler
from statsmodels.tsa.seasonal import seasonal_decompose

import matplotlib.pyplot as plt

%matplotlib inline

# Tensorflow/Keras imports
import tensorflow as tf
from keras.models import Sequential
from keras.callbacks import Callback
from keras.layers import Dense, LSTM

Importing the Data

# Let's load the nvidia stock data into a dataframe using yfinance
nvidia_data_frame = yf.download("NVDA", period="12y", interval="1d")
nvidia_data_frame_viz = nvidia_data_frame.copy()

# Let's extract the day name and day of the week, and add them as columns to the nvidia_data_frame
nvidia_data_frame["day_name"] = nvidia_data_frame.index.day_name()
nvidia_data_frame["day_of_week"] = nvidia_data_frame.index.dayofweek

# Let's take a quick look at the shape of the dataframe
print("NVIDIA stock data shape -->", nvidia_data_frame.shape)
print()

# Let's take a brief look at the contents of the nvidia dataframe
nvidia_data_frame.head(10).style.set_properties(
    **{"background-color": "black", "color": "lawngreen", "border": "1.5px  white"}
)

[*********************100%%**********************]  1 of 1 completed

NVIDIA stock data shape --> (3019, 8)

	Open	High	Low	Close	Adj Close	Volume	day_name	day_of_week
Date
2012-05-02 00:00:00	3.295000	3.307500	3.195000	3.212500	2.946832	68400800	Wednesday	2
2012-05-03 00:00:00	3.220000	3.240000	3.137500	3.157500	2.896382	41953600	Thursday	3
2012-05-04 00:00:00	3.137500	3.155000	3.065000	3.065000	2.811532	54509200	Friday	4
2012-05-07 00:00:00	3.120000	3.172500	3.092500	3.117500	2.859690	60444000	Monday	0
2012-05-08 00:00:00	3.090000	3.135000	3.035000	3.115000	2.857396	46231200	Tuesday	1
2012-05-09 00:00:00	3.060000	3.152500	3.030000	3.127500	2.868862	52060800	Wednesday	2
2012-05-10 00:00:00	3.157500	3.162500	3.055000	3.105000	2.848224	59709200	Thursday	3
2012-05-11 00:00:00	3.362500	3.420000	3.290000	3.302500	3.029390	143514000	Friday	4
2012-05-14 00:00:00	3.275000	3.332500	3.207500	3.282500	3.011045	43747200	Monday	0
2012-05-15 00:00:00	3.282500	3.355000	3.235000	3.250000	2.981232	59994000	Tuesday	1

Gaining Insights from Data

# Let's obtain a brief overview of the nvidia stock dataframe
nvidia_data_frame.info()

<class 'pandas.core.frame.DataFrame'>
DatetimeIndex: 3019 entries, 2012-05-02 to 2024-05-01
Data columns (total 8 columns):
 #   Column       Non-Null Count  Dtype  
---  ------       --------------  -----  
 0   Open         3019 non-null   float64
 1   High         3019 non-null   float64
 2   Low          3019 non-null   float64
 3   Close        3019 non-null   float64
 4   Adj Close    3019 non-null   float64
 5   Volume       3019 non-null   int64  
 6   day_name     3019 non-null   object 
 7   day_of_week  3019 non-null   int32  
dtypes: float64(5), int32(1), int64(1), object(1)
memory usage: 200.5+ KB

# Let's get max range of stocks dataset
delta = nvidia_data_frame.index.max() - nvidia_data_frame.index.min()
print("Time range of stocks dataset --> ", delta)

Time range of stocks dataset -->  4382 days 00:00:00

# Let's see descriptive statistics for all numeric columns
nvidia_data_frame.describe()

	Open	High	Low	Close	Adj Close	Volume	day_of_week
count	3019.000000	3019.000000	3019.000000	3019.000000	3019.000000	3.019000e+03	3019.000000
mean	105.491793	107.405847	103.479915	105.535227	105.266906	4.509939e+07	2.024843
std	159.808699	162.713262	156.569165	159.744254	159.795423	2.457248e+07	1.399036
min	2.872500	2.907500	2.787500	2.845000	2.609724	4.564400e+06	0.000000
25%	5.368750	5.457500	5.276250	5.367500	5.186090	2.938800e+07	1.000000
50%	43.174999	43.639999	42.474998	42.990002	42.587681	3.984680e+07	2.000000
75%	138.778748	140.826248	136.008751	138.275002	137.940536	5.409945e+07	3.000000
max	958.510010	974.000000	935.099976	950.020020	950.020020	3.692928e+08	4.000000

Observations

1. On average, NVIDIA's stock price opened around $124.41 and closed very close to that average at $124.46, indicating there wasn't typically a huge swing in price from the opening to closing on a day-to-day basis.

2. The lowest closing price for NVIDIA's stock was $4.20, which is significantly lower than the average, highlighting that the stock has seen substantial growth since its lowest point.

3. The volume of stocks traded varied widely, but on average about 46.4 million shares were traded each day, with the busiest day seeing over 369 million shares changing hands.

See how many missing data points we have

# Let's get the number of missing data points per column
nvidia_data_frame.isnull().sum()

Open           0
High           0
Low            0
Close          0
Adj Close      0
Volume         0
day_name       0
day_of_week    0
dtype: int64

Missing data points visualization

# Let's set the figure size
plt.figure(figsize=(10, 5), dpi=300)

# Let's visualize the missing points using barplot
msno.bar(nvidia_data_frame_viz.iloc[:, :6])
plt.show()

Data Visualizations(Distributions)

Closing stock price & maximum price during 12 years

# Set the size and background color of the plot to be displayed.
plt.figure(figsize=(18, 7), dpi=300)

# Find the highest closing stock price and the corresponding date over the past 5 years.
high = nvidia_data_frame_viz["Close"].max()
datetime = nvidia_data_frame_viz[nvidia_data_frame_viz["Close"] == high].index.values

# Plot the closing stock prices of Nvidia over time with a specific line color.
plt.plot(nvidia_data_frame_viz.index, nvidia_data_frame_viz["Close"], color="#0f2113")
plt.title(f"Nvidia stock price", fontsize=20)
plt.xlabel("Date", fontsize=15)
plt.ylabel("Daily closing stock price", fontsize=15)

# Locate the highest closing price on the plot and create an annotation.
plt.annotate(
    f"All time high price during\nten year period\nwas ${high}",
    xy=(datetime, high),
    xytext=(datetime, high - 500),
    bbox=dict(boxstyle="round", facecolor="#f5d3bf", edgecolor="#d0d5db"),
    arrowprops=dict(facecolor="#f0190a", headlength=10, shrink=0.05),
)

# Display the plot with all the features set above.
plt.savefig("plots/closing_price_plot.png", dpi=300)
plt.show()

Nvidia - Monthly and Yearly Stock Trends

# Let's get monthly averages of nvidia data_frame
monthly_avg = nvidia_data_frame_viz["Close"].resample("M").mean()

# Let's get yearly averages of nvidia data_frame
yearly_avg = nvidia_data_frame_viz["Close"].resample("A").mean()

# Create a figure and a set of subplots
fig, ax = plt.subplots(
    2, 1, figsize=(10, 8), dpi=300
)  # 2 rows, 1 column, figure size of 10x8 inches

# Monthly averages plot
ax[0].plot(
    monthly_avg.index,
    monthly_avg,
    color="blue",
    marker="o",
    linestyle="-",
    markersize=2,
    alpha=0.7,
)
ax[0].set_title("Monthly Average Closing Price")
ax[0].set_xlabel("Date")
ax[0].set_ylabel("Average Closing Price (USD)")
ax[0].grid(True)

# Yearly averages plot
ax[1].plot(
    yearly_avg.index, yearly_avg, color="green", marker="o", linestyle="-", markersize=4
)
ax[1].set_title("Yearly Average Closing Price")
ax[1].set_xlabel("Date")
ax[1].set_ylabel("Average Closing Price (USD)")
ax[1].grid(True)

# Improve layout and show the plot
plt.tight_layout()
plt.show()

Decomposition of Nvidia Stock Prices into Trend, Seasonality, and Residual Components

# Decomposing the time series using the Close prices
result = seasonal_decompose(
    nvidia_data_frame_viz["Close"], model="additive", period=1
)  # 'period' depends on expected cyclic behavior

# Plotting the decomposed components of the time series
plt.figure(figsize=(18, 10), dpi=300)
plt.subplot(311)
plt.plot(result.trend, label="Trend")
plt.title("Trend Component of Nvidia Stock Prices")
plt.legend(loc="best")

plt.subplot(312)
plt.plot(result.seasonal, label="Seasonality")
plt.title("Seasonal Component of Nvidia Stock Prices")
plt.legend(loc="best")

plt.subplot(313)
plt.plot(result.resid, label="Residuals")
plt.title("Residual Component of Nvidia Stock Prices")
plt.legend(loc="best")

plt.tight_layout()
plt.savefig("plots/trend_seasonality_noise.png")
plt.show()

Investigating Nvidia Stock Prices During Peak Periods in 2022 and 2024

# Let's filter out the data_frame from year 2022 to 204
nvidia_2022_2024 = nvidia_data_frame_viz[
    nvidia_data_frame_viz.index.year.isin([2022, 2024])
]

# Set up the size and layout of the plot
plt.figure(figsize=(14, 6), dpi=300)
# Plot the closing prices on the plot, with a label for identification
plt.plot(nvidia_2022_2024["Close"], label="Close Price")
# # Plot the highest prices of the day on the plot, with a label for identification
# plt.plot(nvidia_2022_2024['High'], label='High Price')


# Add a title to the plot, which gives a general idea of what the data represents
plt.title("Nvidia Stock Prices for 2022 and 2024")
# Label the x-axis as 'Date' which shows the timeline
plt.xlabel("Date")
# Label the y-axis as 'Price' to indicate what the numbers on the y-axis represent
plt.ylabel("Price")
plt.legend()

# Add a grid to the background of the plot to improve readability of the graph
plt.grid(True)
plt.show()

Investigating Nvidia Stock Prices During Peak Periods in 2023 and 2024

# Let's filter out the data_frame from year 2022 to 204
nvidia_2023_2024 = nvidia_data_frame_viz[
    nvidia_data_frame_viz.index.year.isin([2023, 2024])
]

# Set up the size and layout of the plot
plt.figure(figsize=(14, 6), dpi=300)
# Plot the closing prices on the plot, with a label for identification
plt.plot(nvidia_2023_2024["Close"], label="Close Price")
# # Plot the highest prices of the day on the plot, with a label for identification
# plt.plot(nvidia_2022_2024['High'], label='High Price')


# Add a title to the plot, which gives a general idea of what the data represents
plt.title("Nvidia Stock Prices for 2023 and 2024")
# Label the x-axis as 'Date' which shows the timeline
plt.xlabel("Date")
# Label the y-axis as 'Price' to indicate what the numbers on the y-axis represent
plt.ylabel("Price")
plt.legend()

# Add a grid to the background of the plot to improve readability of the graph
# plt.grid(True)
plt.savefig("plots/2023_2024_stock_price.png")
plt.show()

Trade volume of Nvidia stock over a period of 2014-2024

# Calculate the average, maximum, and minimum trade volume.
ave_y = nvidia_data_frame_viz["Volume"].mean()
max_y = nvidia_data_frame_viz["Volume"].max()
min_y = nvidia_data_frame_viz["Volume"].min()

# Set the figure size and background color.
plt.figure(figsize=(18, 7), dpi=300)

# Plot the trade volume data.
plt.plot(
    nvidia_data_frame_viz.index, nvidia_data_frame_viz["Volume"], color="#283954"
)  # Using the index as the date.
plt.title(f"Nvidia stock trade volume", fontsize=20)
plt.xlabel("Date", fontsize=15)
plt.ylabel("Daily trade volume", fontsize=15)

# Add horizontal lines for max, min, and average trade volumes.
plt.axhline(y=max_y, linestyle="--", lw=2.2, color="green")
plt.axhline(y=min_y, linestyle="--", lw=2.2, color="red")
plt.axhline(y=ave_y, linestyle="--", lw=2.8, color="yellow")

# Find the date for the maximum trade volume.
max_vol_date = nvidia_data_frame_viz[nvidia_data_frame_viz["Volume"] == max_y].index[0]

# Add a vertical line for the date with maximum trade volume.
plt.axvline(x=max_vol_date, linestyle="--", lw=2.2, color="#0aebff")

#Add annotations for average, maximum, and minimum trade volumes.
plt.annotate(
    f"Average trade volume {round(ave_y,2)}",
    xy=(max_vol_date, ave_y),
    xytext=(max_vol_date, ave_y + 10000000),
    bbox=dict(boxstyle="round", facecolor="#e8e0ba", edgecolor="#d0d5db"),
    arrowprops=dict(facecolor="#333333", shrink=0.05),
)

plt.annotate(
    f"Maximum trade volume {max_y}",
    xy=(max_vol_date, max_y),
    xytext=(max_vol_date, max_y - 10000000),
    bbox=dict(boxstyle="round", facecolor="#e8e0ba", edgecolor="#d0d5db"),
    arrowprops=dict(facecolor="#333333", shrink=0.05),
)

plt.annotate(
    f"Minimum trade volume {min_y}",
    xy=(max_vol_date, min_y),
    xytext=(max_vol_date, min_y + 10000000),
    bbox=dict(boxstyle="round", facecolor="#e8e0ba", edgecolor="#d0d5db"),
    arrowprops=dict(facecolor="#333333", shrink=0.05),
)

# Display the plot.
plt.savefig("plots/trade_volume.png")
plt.show()

Moving average of the Nvidia stock

# List of moving average days
ma_day = [10, 20, 50]

# Compute moving averages for each specified day count in Nvidia_data_frame
for ma in ma_day:
    column_name = f"MA for {ma} days"
    nvidia_data_frame_viz[column_name] = (
        nvidia_data_frame_viz["Adj Close"].rolling(ma).mean()
    )

# Create a subplot figure for the moving averages
fig, ax = plt.subplots(dpi=300)
fig.set_figheight(7)
fig.set_figwidth(18)

# Plot the adjusted close price and moving averages
nvidia_data_frame_viz[
    ["Adj Close", "MA for 10 days", "MA for 20 days", "MA for 50 days"]
].plot(ax=ax)
ax.set_title("NVIDIA Moving Averages")

# Display the plot
plt.tight_layout()

Daily return of the Nvidia stock on average

# Calculate the percent change for each day to find daily returns.
nvidia_data_frame_viz["Daily Return"] = nvidia_data_frame_viz["Adj Close"].pct_change()

# Create a single plot for the daily returns of NVIDIA.
fig, ax = plt.subplots(dpi=300)
fig.set_figheight(7)
fig.set_figwidth(20)

# Plot the daily return percentage with specific styling.
nvidia_data_frame_viz["Daily Return"].plot(ax=ax, legend=True, linestyle="--")
ax.set_title("NVIDIA Daily Returns")

# Properly layout the plot to handle subplots if necessary.
fig.tight_layout()

# Display the plot.
plt.savefig("plots/daily_returns.png")
plt.tight_layout()

Candlestick charts of Nvidia stock

# Create a candlestick chart trace for NVIDIA using its stock data.
trace = go.Candlestick(
    x=nvidia_data_frame_viz.index,  # Use the DataFrame index for date
    open=nvidia_data_frame_viz["Open"],  # Opening price column
    high=nvidia_data_frame_viz["High"],  # Highest price column
    low=nvidia_data_frame_viz["Low"],  # Lowest price column
    close=nvidia_data_frame_viz["Close"],
)  # Closing price column

# Define the layout for the candlestick chart.
layout = {
    "title": f"NVIDIA Historical Stock Price",  # Title with NVIDIA's name
    "xaxis": {"title": "Date"},  # X-axis label
    "yaxis": {"title": "Stock Price (USD$)"},  # Y-axis label
}

# Combine the trace into a list to form the complete plot data.
data = [trace]

# Create the figure object with the data and layout defined above.
fig = go.Figure(data=data, layout=layout)
fig.update_layout(
    width=1100,  # Set the width of the figure (in pixels)
    height=600,  # Set the height of the figure (in pixels)
    xaxis_rangeslider_visible=True,
)
fig.show()  # Display the plot

Data Cleaning

# Let's drop the unnecessary columns such as Open, High, Low, Adj Close, and Volume,
# since our objective is to predict NVIDIA's future closing price.
nvidia_data_frame.drop(
    ["Open", "High", "Low", "Adj Close", "Volume", "day_name"], axis=1, inplace=True
)
# Let's take a quick look at the shape of the dataframe
print("NVIDIA stock data shape -->", nvidia_data_frame.shape)
print()

# Let's take a brief look at the contents of the nvidia dataframe
nvidia_data_frame.head(5).style.set_properties(
    **{"background-color": "black", "color": "lawngreen", "border": "1.5px  white"}
)

NVIDIA stock data shape --> (3019, 2)

	Close	day_of_week
Date
2012-05-02 00:00:00	3.212500	2
2012-05-03 00:00:00	3.157500	3
2012-05-04 00:00:00	3.065000	4
2012-05-07 00:00:00	3.117500	0
2012-05-08 00:00:00	3.115000	1

Data Normalization

closing_price = nvidia_data_frame.filter(["Close"])

# Convert the 'Close' and 'day_of_week' column to a numpy array
closing_price_data = closing_price.values
day_of_week_data = nvidia_data_frame[["day_of_week"]].values

# Let's create a MinMaxScaler object with a feature range from 0 to 1
scaler = MinMaxScaler(feature_range=(0, 1))
scaler_day_of_week = MinMaxScaler(feature_range=(0, 1))

# Scale the dataset to fit within the feature range of 0 to 1 and assign the result to scaled_data
nvidia_data_frame.loc[:, "Close"] = scaler.fit_transform(closing_price)
nvidia_data_frame["day_of_week"] = scaler_day_of_week.fit_transform(day_of_week_data)

# Let's see few values of the scaled data
nvidia_data_frame.head(5).style.set_properties(
    **{"background-color": "black", "color": "lawngreen", "border": "1.5px  white"}
)

	Close	day_of_week
Date
2012-05-02 00:00:00	0.000388	0.500000
2012-05-03 00:00:00	0.000330	0.750000
2012-05-04 00:00:00	0.000232	1.000000
2012-05-07 00:00:00	0.000288	0.000000
2012-05-08 00:00:00	0.000285	0.250000

Descriptive statisctics after normalization

# Let's see descriptive statistics for all numeric columns after normalization
nvidia_data_frame.describe()

	Close	day_of_week
count	3019.000000	3019.000000
mean	0.108417	0.506211
std	0.168653	0.349759
min	0.000000	0.000000
25%	0.002663	0.250000
50%	0.042384	0.500000
75%	0.142983	0.750000
max	1.000000	1.000000

Data Preparation

# Function to create sequences of 90/180 timesteps as inputs with the corresponding outputs
def create_windowed_df(data, period):
    # Initialize empty lists to hold the input and output parts of the training data
    X = []
    y = []
    # Loop over the training data to create sequences of 90/180 timesteps as inputs with the corresponding outputs
    for i in range(period, len(data)):
        X.append(
            data[i - period : i, 0]
        )  # Append the past 90/180 values as a single input sequence
        y.append(
            data[i, 0]
        )  # Append the current value as the output for the input sequence
    return X, y

# Get the number of rows to train the model on
training_data_len = int(np.ceil(len(nvidia_data_frame) * 0.8))
validation_data_len = int(np.ceil(len(nvidia_data_frame) * 0.9))


# Let's print the training data length
print("Length of Training Data -->", training_data_len)
print("Length of Validation Data -->", validation_data_len - training_data_len)
print("Lenagth of Test Data -->", len(nvidia_data_frame) - validation_data_len)

Length of Training Data --> 2416
Length of Validation Data --> 302
Lenagth of Test Data --> 301

Data Splitting (3 months Window)

# Extract the whole training subset from the Nvidia data frame
overall_data = nvidia_data_frame.values

overall_x_train_3_months, overall_y_train_3_months = create_windowed_df(
    overall_data, period=90
)

# Convert the list of input sequences and outputs into numpy arrays for machine learning processing
overall_x_train_3_months, overall_y_train_3_months = np.array(
    overall_x_train_3_months
), np.array(overall_y_train_3_months)

# Reshape the input data to fit the model's expected input shape (samples, timesteps, features)
overall_x_train_3_months = np.reshape(
    overall_x_train_3_months,
    (overall_x_train_3_months.shape[0], overall_x_train_3_months.shape[1], 1),
)

# Extract the whole training subset from the Nvidia data frame
train_data_3_months = overall_data[0 : int(training_data_len)]

x_train_3_months, y_train_3_months = create_windowed_df(train_data_3_months, period=90)

# Convert the list of input sequences and outputs into numpy arrays for machine learning processing
x_train_3_months, y_train_3_months = np.array(x_train_3_months), np.array(
    y_train_3_months
)

# Reshape the input data to fit the model's expected input shape (samples, timesteps, features)
x_train_3_months = np.reshape(
    x_train_3_months, (x_train_3_months.shape[0], x_train_3_months.shape[1], 1)
)

# Extract the whole training subset from the Nvidia data frame
validation_data_3_months = overall_data[
    int(training_data_len) : int(validation_data_len)
]

x_valid_3_months, y_valid_3_months = create_windowed_df(
    validation_data_3_months, period=90
)

# Convert the list of input sequences and outputs into numpy arrays for machine learning processing
x_valid_3_months, y_valid_3_months = np.array(x_valid_3_months), np.array(
    y_valid_3_months
)

# Reshape the input data to fit the model's expected input shape (samples, timesteps, features)
x_valid_3_months = np.reshape(
    x_valid_3_months, (x_valid_3_months.shape[0], x_valid_3_months.shape[1], 1)
)

# Extract the whole training subset from the Nvidia data frame
test_data_3_months = overall_data[int(validation_data_len) :]

x_test_3_months, y_test_3_months = create_windowed_df(test_data_3_months, period=90)

# Convert the list of input sequences and outputs into numpy arrays for machine learning processing
x_test_3_months, y_test_3_months = np.array(x_test_3_months), np.array(y_test_3_months)

# Reshape the input data to fit the model's expected input shape (samples, timesteps, features)
x_test_3_months = np.reshape(
    x_test_3_months, (x_test_3_months.shape[0], x_test_3_months.shape[1], 1)
)

Data Splitting (6 months Window)

# Extract the whole training subset from the Nvidia data frame
overall_data = nvidia_data_frame.values

overall_x_train_6_months, overall_y_train_6_months = create_windowed_df(
    overall_data, period=180
)

# Convert the list of input sequences and outputs into numpy arrays for machine learning processing
overall_x_train_6_months, overall_y_train_6_months = np.array(
    overall_x_train_6_months
), np.array(overall_y_train_6_months)

# Reshape the input data to fit the model's expected input shape (samples, timesteps, features)
overall_x_train_6_months = np.reshape(
    overall_x_train_6_months,
    (overall_x_train_6_months.shape[0], overall_x_train_6_months.shape[1], 1),
)

# Extract the whole training subset from the Nvidia data frame
train_data_6_months = overall_data[0 : int(training_data_len)]

x_train_6_months, y_train_6_months = create_windowed_df(train_data_6_months, period=180)

# Convert the list of input sequences and outputs into numpy arrays for machine learning processing
x_train_6_months, y_train_6_months = np.array(x_train_6_months), np.array(
    y_train_6_months
)

# Reshape the input data to fit the model's expected input shape (samples, timesteps, features)
x_train_6_months = np.reshape(
    x_train_6_months, (x_train_6_months.shape[0], x_train_6_months.shape[1], 1)
)

# Extract the whole training subset from the Nvidia data frame
validation_data_6_months = overall_data[
    int(training_data_len) : int(validation_data_len)
]

x_valid_6_months, y_valid_6_months = create_windowed_df(
    validation_data_6_months, period=180
)

# Convert the list of input sequences and outputs into numpy arrays for machine learning processing
x_valid_6_months, y_valid_6_months = np.array(x_valid_6_months), np.array(
    y_valid_6_months
)

# Reshape the input data to fit the model's expected input shape (samples, timesteps, features)
x_valid_6_months = np.reshape(
    x_valid_6_months, (x_valid_6_months.shape[0], x_valid_6_months.shape[1], 1)
)

# Extract the whole training subset from the Nvidia data frame
test_data_6_months = overall_data[int(validation_data_len) :]

x_test_6_months, y_test_6_months = create_windowed_df(test_data_6_months, period=180)

# Convert the list of input sequences and outputs into numpy arrays for machine learning processing
x_test_6_months, y_test_6_months = np.array(x_test_6_months), np.array(y_test_6_months)

# Reshape the input data to fit the model's expected input shape (samples, timesteps, features)
x_test_6_months = np.reshape(
    x_test_6_months, (x_test_6_months.shape[0], x_test_6_months.shape[1], 1)
)

Let's see shapes of validation and test dataset(3 month window)

print("Shape of input Training Data(3 month window)   -->", x_train_3_months.shape)
print("Shape of input Validation Data(3 month window) -->", x_valid_3_months.shape)
print("Shape of input Test Data (3 month window)      -->", x_test_3_months.shape)

Shape of input Training Data(3 month window)   --> (2326, 90, 1)
Shape of input Validation Data(3 month window) --> (212, 90, 1)
Shape of input Test Data (3 month window)      --> (211, 90, 1)

print("Shape of output Training Data(3 month window)   -->", y_train_3_months.shape)
print("Shape of output Validation Data(3 month window) -->", y_valid_3_months.shape)
print("Shape of output Test Data(3 month window)       -->", y_test_3_months.shape)

Shape of output Training Data(3 month window)   --> (2326,)
Shape of output Validation Data(3 month window) --> (212,)
Shape of output Test Data(3 month window)       --> (211,)

Let's see shapes of validation and test dataset(6 month window)

print("Shape of input Training Data(3 month window)   -->", x_train_6_months.shape)
print("Shape of input Validation Data(3 month window) -->", x_valid_6_months.shape)
print("Shape of input Test Data (3 month window)      -->", x_test_6_months.shape)

Shape of input Training Data(3 month window)   --> (2236, 180, 1)
Shape of input Validation Data(3 month window) --> (122, 180, 1)
Shape of input Test Data (3 month window)      --> (121, 180, 1)

print("Shape of output Training Data(3 month window)   -->", y_train_6_months.shape)
print("Shape of output Validation Data(3 month window) -->", y_valid_6_months.shape)
print("Shape of output Test Data(3 month window)       -->", y_test_6_months.shape)

Shape of output Training Data(3 month window)   --> (2236,)
Shape of output Validation Data(3 month window) --> (122,)
Shape of output Test Data(3 month window)       --> (121,)

Build a model to overfit on the entire dataset

# Function to plot the loss and mean absolute error
def plot_loss_and_mae(model_name, model_history):
    figure, (ax1, ax2) = plt.subplots(
        1, 2, figsize=(18, 6), dpi=300
    )  # Define a figure with two subplots
    figure.suptitle(model_name)  # Set the title of the figure

    # Plot the loss over epochs on the first subplot
    ax1.plot(
        model_history.history["loss"], label="Loss", color="blue"
    )  # Plot the loss history
    ax1.set_title("Training Loss")  # Title for the loss plot
    ax1.set_xlabel("Epoch")  # Label for the x-axis
    ax1.set_ylabel("Loss")  # Label for the y-axis
    ax1.legend(loc="upper right")  # Legend positioned in the upper right

    # Plot the mean absolute error over epochs on the second subplot
    ax2.plot(
        model_history.history["mean_absolute_error"],
        label="Mean Absolute Error",
        color="green",
    )  # Plot the MAE history
    ax2.set_title("Training Mean Absolute Error")  # Title for the MAE plot
    ax2.set_xlabel("Epoch")  # Label for the x-axis
    ax2.set_ylabel("Mean Absolute Error")  # Label for the y-axis
    ax2.legend(loc="upper right")  # Legend positioned in the upper right
    plt.savefig("plots/best_overfitting_arch.png")
    plt.show()  # Show the plot

class PrintMetricsCallback(Callback):
    def __init__(self, print_validation=False):
        super().__init__()
        self.print_validation = print_validation

    # Define a callback to print metrics at the end of each epoch
    def on_epoch_end(self, epoch, logs=None):
        # Check if the epoch number plus one is divisible by 10 to print every 10 epochs, or it's the first epoch
        if (epoch + 1) % 10 == 0 or epoch == 0:
            output = f"Epoch {epoch + 1}: Loss = {logs['loss']:.4f}, MAE = {logs.get('mean_absolute_error', 'N/A'):.4f}"
            if self.print_validation:
                # Append validation metrics to the output string if flag is true
                output += f", Val Loss = {logs.get('val_loss', 'N/A'):.4f}, Val MAE = {logs.get('val_mean_absolute_error', 'N/A'):.4f}"
            print(output)

Model Building with one LSTM layer and one Dense Layer (8-1) and 30 epochs 3 months window

# Let's create a keras sequential model
model_8_1_30_3_months_overfit = Sequential()

# Let's add LSTM layer to the model network
model_8_1_30_3_months_overfit.add(
    LSTM(8, return_sequences=False, input_shape=(x_train_3_months.shape[1], 1))
)
# Let's add dense layer to the model network
model_8_1_30_3_months_overfit.add(Dense(1))

# Let's build the model using mean_squared_error as the loss function
# and mean_absolute_error as the evaluation metric during the compilation process
model_8_1_30_3_months_overfit.compile(
    optimizer="adam", loss="mean_squared_error", metrics=["mean_absolute_error"]
)

# Let's fit the sequential model with input features and output label
history_8_1_30_3_months_overfit = model_8_1_30_3_months_overfit.fit(
    overall_x_train_3_months, overall_y_train_3_months, batch_size=1, epochs=30
)

Epoch 1/30
2929/2929 [==============================] - 21s 7ms/step - loss: 4.6071e-04 - mean_absolute_error: 0.0095
Epoch 2/30
2929/2929 [==============================] - 20s 7ms/step - loss: 1.3417e-04 - mean_absolute_error: 0.0069
Epoch 3/30
2929/2929 [==============================] - 20s 7ms/step - loss: 1.1416e-04 - mean_absolute_error: 0.0064
Epoch 4/30
2929/2929 [==============================] - 20s 7ms/step - loss: 9.7010e-05 - mean_absolute_error: 0.0059
Epoch 5/30
2929/2929 [==============================] - 20s 7ms/step - loss: 9.9265e-05 - mean_absolute_error: 0.0057
Epoch 6/30
2929/2929 [==============================] - 20s 7ms/step - loss: 9.2572e-05 - mean_absolute_error: 0.0056
Epoch 7/30
2929/2929 [==============================] - 20s 7ms/step - loss: 7.8645e-05 - mean_absolute_error: 0.0052
Epoch 8/30
2929/2929 [==============================] - 20s 7ms/step - loss: 8.0537e-05 - mean_absolute_error: 0.0052
Epoch 9/30
2929/2929 [==============================] - 20s 7ms/step - loss: 7.4794e-05 - mean_absolute_error: 0.0051
Epoch 10/30
2929/2929 [==============================] - 19s 7ms/step - loss: 7.4594e-05 - mean_absolute_error: 0.0050
Epoch 11/30
2929/2929 [==============================] - 20s 7ms/step - loss: 6.6983e-05 - mean_absolute_error: 0.0047
Epoch 12/30
2929/2929 [==============================] - 20s 7ms/step - loss: 6.8448e-05 - mean_absolute_error: 0.0048
Epoch 13/30
2929/2929 [==============================] - 20s 7ms/step - loss: 6.8898e-05 - mean_absolute_error: 0.0046
Epoch 14/30
2929/2929 [==============================] - 19s 7ms/step - loss: 7.0316e-05 - mean_absolute_error: 0.0048
Epoch 15/30
2929/2929 [==============================] - 20s 7ms/step - loss: 6.4307e-05 - mean_absolute_error: 0.0046
Epoch 16/30
2929/2929 [==============================] - 19s 7ms/step - loss: 7.0029e-05 - mean_absolute_error: 0.0049
Epoch 17/30
2929/2929 [==============================] - 20s 7ms/step - loss: 6.4745e-05 - mean_absolute_error: 0.0045
Epoch 18/30
2929/2929 [==============================] - 19s 7ms/step - loss: 6.3323e-05 - mean_absolute_error: 0.0046
Epoch 19/30
2929/2929 [==============================] - 20s 7ms/step - loss: 6.7371e-05 - mean_absolute_error: 0.0046
Epoch 20/30
2929/2929 [==============================] - 20s 7ms/step - loss: 6.8580e-05 - mean_absolute_error: 0.0047
Epoch 21/30
2929/2929 [==============================] - 20s 7ms/step - loss: 6.3964e-05 - mean_absolute_error: 0.0044
Epoch 22/30
2929/2929 [==============================] - 20s 7ms/step - loss: 6.3956e-05 - mean_absolute_error: 0.0046
Epoch 23/30
2929/2929 [==============================] - 20s 7ms/step - loss: 6.2620e-05 - mean_absolute_error: 0.0044
Epoch 24/30
2929/2929 [==============================] - 20s 7ms/step - loss: 6.2219e-05 - mean_absolute_error: 0.0044
Epoch 25/30
2929/2929 [==============================] - 20s 7ms/step - loss: 6.2376e-05 - mean_absolute_error: 0.0045
Epoch 26/30
2929/2929 [==============================] - 20s 7ms/step - loss: 5.9327e-05 - mean_absolute_error: 0.0044
Epoch 27/30
2929/2929 [==============================] - 22s 7ms/step - loss: 6.0429e-05 - mean_absolute_error: 0.0042
Epoch 28/30
2929/2929 [==============================] - 20s 7ms/step - loss: 6.2936e-05 - mean_absolute_error: 0.0045
Epoch 29/30
2929/2929 [==============================] - 20s 7ms/step - loss: 6.2701e-05 - mean_absolute_error: 0.0045
Epoch 30/30
2929/2929 [==============================] - 20s 7ms/step - loss: 5.8642e-05 - mean_absolute_error: 0.0043

# Let's plot the mean_squared_error and mean_absolute_error using model history on the entire dataset
plot_loss_and_mae(
    model_name="model_8_1_30_3_months_overfit",
    model_history=history_8_1_30_3_months_overfit,
)

Observations

Using three months of data as a window to predict the stock price, the mean absolute error is decreasing gradually; one option to potentially decrease it further is to enhance the model's complexity by adding more or removing neurons/layers.

• 1 LSTM(8) 1 Dense(1) - 3 months
• Epochs: 30
• Batch Size: 1
• Training Loss: 5.8642e-05
• Mean Absolute Error: 0.0043

Model Building with one LSTM layer and one Dense Layer (8-1) and 30 epochs 6 months window

# Let's create a keras sequential model
model_8_1_30_6_months_overfit = Sequential()

# Let's add LSTM layer to the model network
model_8_1_30_6_months_overfit.add(
    LSTM(8, return_sequences=False, input_shape=(x_train_6_months.shape[1], 1))
)
# Let's add dense layer to the model network
model_8_1_30_6_months_overfit.add(Dense(1))

# Let's build the model using mean_squared_error as the loss function
# and mean_absolute_error as the evaluation metric during the compilation process
model_8_1_30_6_months_overfit.compile(
    optimizer="adam", loss="mean_squared_error", metrics=["mean_absolute_error"]
)

# Let's fit the sequential model with input features and output label
history_8_1_30_6_months_overfit = model_8_1_30_6_months_overfit.fit(
    overall_x_train_6_months, overall_y_train_6_months, batch_size=1, epochs=30
)

Epoch 1/30
2839/2839 [==============================] - 38s 13ms/step - loss: 0.0045 - mean_absolute_error: 0.0246
Epoch 2/30
2839/2839 [==============================] - 37s 13ms/step - loss: 2.9035e-04 - mean_absolute_error: 0.0106
Epoch 3/30
2839/2839 [==============================] - 37s 13ms/step - loss: 1.9972e-04 - mean_absolute_error: 0.0085
Epoch 4/30
2839/2839 [==============================] - 37s 13ms/step - loss: 1.5518e-04 - mean_absolute_error: 0.0076
Epoch 5/30
2839/2839 [==============================] - 37s 13ms/step - loss: 1.3069e-04 - mean_absolute_error: 0.0069
Epoch 6/30
2839/2839 [==============================] - 37s 13ms/step - loss: 1.1599e-04 - mean_absolute_error: 0.0065
Epoch 7/30
2839/2839 [==============================] - 37s 13ms/step - loss: 9.8349e-05 - mean_absolute_error: 0.0060
Epoch 8/30
2839/2839 [==============================] - 37s 13ms/step - loss: 9.4472e-05 - mean_absolute_error: 0.0059
Epoch 9/30
2839/2839 [==============================] - 37s 13ms/step - loss: 9.0597e-05 - mean_absolute_error: 0.0056
Epoch 10/30
2839/2839 [==============================] - 38s 13ms/step - loss: 8.4802e-05 - mean_absolute_error: 0.0054
Epoch 11/30
2839/2839 [==============================] - 37s 13ms/step - loss: 7.7434e-05 - mean_absolute_error: 0.0052
Epoch 12/30
2839/2839 [==============================] - 37s 13ms/step - loss: 7.8579e-05 - mean_absolute_error: 0.0052
Epoch 13/30
2839/2839 [==============================] - 37s 13ms/step - loss: 6.8393e-05 - mean_absolute_error: 0.0048
Epoch 14/30
2839/2839 [==============================] - 37s 13ms/step - loss: 7.1462e-05 - mean_absolute_error: 0.0049
Epoch 15/30
2839/2839 [==============================] - 37s 13ms/step - loss: 6.7460e-05 - mean_absolute_error: 0.0047
Epoch 16/30
2839/2839 [==============================] - 37s 13ms/step - loss: 6.6729e-05 - mean_absolute_error: 0.0047
Epoch 17/30
2839/2839 [==============================] - 37s 13ms/step - loss: 6.3276e-05 - mean_absolute_error: 0.0045
Epoch 18/30
2839/2839 [==============================] - 37s 13ms/step - loss: 6.3032e-05 - mean_absolute_error: 0.0045
Epoch 19/30
2839/2839 [==============================] - 37s 13ms/step - loss: 6.2422e-05 - mean_absolute_error: 0.0045
Epoch 20/30
2839/2839 [==============================] - 36s 13ms/step - loss: 5.8113e-05 - mean_absolute_error: 0.0044
Epoch 21/30
2839/2839 [==============================] - 37s 13ms/step - loss: 6.0146e-05 - mean_absolute_error: 0.0044
Epoch 22/30
2839/2839 [==============================] - 36s 13ms/step - loss: 6.3374e-05 - mean_absolute_error: 0.0042
Epoch 23/30
2839/2839 [==============================] - 37s 13ms/step - loss: 5.6410e-05 - mean_absolute_error: 0.0042
Epoch 24/30
2839/2839 [==============================] - 37s 13ms/step - loss: 5.9206e-05 - mean_absolute_error: 0.0043
Epoch 25/30
2839/2839 [==============================] - 37s 13ms/step - loss: 6.1812e-05 - mean_absolute_error: 0.0045
Epoch 26/30
2839/2839 [==============================] - 37s 13ms/step - loss: 5.7169e-05 - mean_absolute_error: 0.0042
Epoch 27/30
2839/2839 [==============================] - 37s 13ms/step - loss: 6.1413e-05 - mean_absolute_error: 0.0044
Epoch 28/30
2839/2839 [==============================] - 37s 13ms/step - loss: 5.9653e-05 - mean_absolute_error: 0.0043
Epoch 29/30
2839/2839 [==============================] - 37s 13ms/step - loss: 6.0017e-05 - mean_absolute_error: 0.0044
Epoch 30/30
2839/2839 [==============================] - 37s 13ms/step - loss: 5.8490e-05 - mean_absolute_error: 0.0044

# Let's plot the mean_squared_error and mean_absolute_error using model history on the entire dataset
plot_loss_and_mae(
    model_name="model_8_1_30_6_months_overfit",
    model_history=history_8_1_30_6_months_overfit,
)

Observations

Using six months of data as a window to predict the stock price, the mean absolute error is decreasing gradually and is almost similar to the model trained with window data from the past three months. One option to potentially decrease it further is to enhance the model's complexity by adding or removing more layers or neurons.

• 1 LSTM(8) 1 Dense(1) - 6 months
• Epochs: 30
• Batch Size: 1
• Training Loss: 5.8490e-05
• Mean Absolute Error: 0.0044

Model Building with one LSTM layer and one Dense Layer (64-1) and 30 epochs 3 months window

# Let's create a keras sequential model
model_64_1_30_3_months_overfit = Sequential()

# Let's add LSTM layer to the model network
model_64_1_30_3_months_overfit.add(
    LSTM(64, return_sequences=False, input_shape=(x_train_3_months.shape[1], 1))
)
# Let's add dense layer to the model network
model_64_1_30_3_months_overfit.add(Dense(1))

# Let's build the model using mean_squared_error as the loss function
# and mean_absolute_error as the evaluation metric during the compilation process
model_64_1_30_3_months_overfit.compile(
    optimizer="adam", loss="mean_squared_error", metrics=["mean_absolute_error"]
)

# Let's fit the sequential model with input features and output label
history_64_1_30_3_months_overfit = model_64_1_30_3_months_overfit.fit(
    overall_x_train_3_months, overall_y_train_3_months, batch_size=1, epochs=30
)

Epoch 1/30
2929/2929 [==============================] - 25s 8ms/step - loss: 4.0641e-04 - mean_absolute_error: 0.0103
Epoch 2/30
2929/2929 [==============================] - 23s 8ms/step - loss: 2.2345e-04 - mean_absolute_error: 0.0087
Epoch 3/30
2929/2929 [==============================] - 23s 8ms/step - loss: 1.8468e-04 - mean_absolute_error: 0.0074
Epoch 4/30
2929/2929 [==============================] - 23s 8ms/step - loss: 1.5090e-04 - mean_absolute_error: 0.0068
Epoch 5/30
2929/2929 [==============================] - 23s 8ms/step - loss: 1.1301e-04 - mean_absolute_error: 0.0059
Epoch 6/30
2929/2929 [==============================] - 23s 8ms/step - loss: 1.1173e-04 - mean_absolute_error: 0.0058
Epoch 7/30
2929/2929 [==============================] - 23s 8ms/step - loss: 9.7083e-05 - mean_absolute_error: 0.0056
Epoch 8/30
2929/2929 [==============================] - 23s 8ms/step - loss: 9.4722e-05 - mean_absolute_error: 0.0055
Epoch 9/30
2929/2929 [==============================] - 23s 8ms/step - loss: 9.5371e-05 - mean_absolute_error: 0.0054
Epoch 10/30
2929/2929 [==============================] - 24s 8ms/step - loss: 1.0522e-04 - mean_absolute_error: 0.0054
Epoch 11/30
2929/2929 [==============================] - 24s 8ms/step - loss: 9.4873e-05 - mean_absolute_error: 0.0051
Epoch 12/30
2929/2929 [==============================] - 24s 8ms/step - loss: 7.8796e-05 - mean_absolute_error: 0.0049
Epoch 13/30
2929/2929 [==============================] - 24s 8ms/step - loss: 7.4228e-05 - mean_absolute_error: 0.0049
Epoch 14/30
2929/2929 [==============================] - 23s 8ms/step - loss: 7.7876e-05 - mean_absolute_error: 0.0048
Epoch 15/30
2929/2929 [==============================] - 23s 8ms/step - loss: 6.8480e-05 - mean_absolute_error: 0.0047
Epoch 16/30
2929/2929 [==============================] - 23s 8ms/step - loss: 1.7266e-04 - mean_absolute_error: 0.0048
Epoch 17/30
2929/2929 [==============================] - 23s 8ms/step - loss: 6.7012e-05 - mean_absolute_error: 0.0046
Epoch 18/30
2929/2929 [==============================] - 24s 8ms/step - loss: 7.4167e-05 - mean_absolute_error: 0.0049
Epoch 19/30
2929/2929 [==============================] - 24s 8ms/step - loss: 9.0091e-05 - mean_absolute_error: 0.0050
Epoch 20/30
2929/2929 [==============================] - 24s 8ms/step - loss: 7.4273e-05 - mean_absolute_error: 0.0046
Epoch 21/30
2929/2929 [==============================] - 23s 8ms/step - loss: 7.1237e-05 - mean_absolute_error: 0.0045
Epoch 22/30
2929/2929 [==============================] - 23s 8ms/step - loss: 7.7773e-05 - mean_absolute_error: 0.0048
Epoch 23/30
2929/2929 [==============================] - 24s 8ms/step - loss: 7.7650e-05 - mean_absolute_error: 0.0046
Epoch 24/30
2929/2929 [==============================] - 27s 9ms/step - loss: 6.4758e-05 - mean_absolute_error: 0.0044
Epoch 25/30
2929/2929 [==============================] - 24s 8ms/step - loss: 6.9461e-05 - mean_absolute_error: 0.0046
Epoch 26/30
2929/2929 [==============================] - 23s 8ms/step - loss: 7.3474e-05 - mean_absolute_error: 0.0045
Epoch 27/30
2929/2929 [==============================] - 28s 10ms/step - loss: 6.2413e-05 - mean_absolute_error: 0.0044
Epoch 28/30
2929/2929 [==============================] - 23s 8ms/step - loss: 6.3835e-05 - mean_absolute_error: 0.0042
Epoch 29/30
2929/2929 [==============================] - 23s 8ms/step - loss: 6.4746e-05 - mean_absolute_error: 0.0044
Epoch 30/30
2929/2929 [==============================] - 23s 8ms/step - loss: 6.4091e-05 - mean_absolute_error: 0.0044

# Let's plot the mean_squared_error and mean_absolute_error using model history on the entire dataset
plot_loss_and_mae(
    model_name="model_64_1_30_3_months_overfit",
    model_history=history_64_1_30_3_months_overfit,
)

Observations

Using three months of data as a window to predict the stock price, the mean absolute error is gradually decreasing; however, at the 15th epoch, the training loss observed a spike, and the mean absolute error is almost similar to that observed with a single LSTM layer with 8 neurons. To potentially further reduce the error, one option is to enhance the model's complexity by adding or removing layers or neurons.

• 1 LSTM(64) 1 Dense(1) - 3 months
• Epochs: 30
• Batch Size: 1
• Training Loss: 6.4091e-05
• Mean Absolute Error: 0.0044

Model Building with one LSTM layer and one Dense Layer (64-1) and 30 epochs 6 months window

# Let's create a keras sequential model
model_64_1_30_6_months_overfit = Sequential()

# Let's add LSTM layer to the model network
model_64_1_30_6_months_overfit.add(
    LSTM(64, return_sequences=False, input_shape=(x_train_6_months.shape[1], 1))
)
# Let's add dense layer to the model network
model_64_1_30_6_months_overfit.add(Dense(1))

# Let's build the model using mean_squared_error as the loss function
# and mean_absolute_error as the evaluation metric during the compilation process
model_64_1_30_6_months_overfit.compile(
    optimizer="adam", loss="mean_squared_error", metrics=["mean_absolute_error"]
)

# Let's fit the sequential model with input features and output label
history_64_1_30_6_months_overfit = model_64_1_30_6_months_overfit.fit(
    overall_x_train_6_months, overall_y_train_6_months, batch_size=1, epochs=30
)

Epoch 1/30
2839/2839 [==============================] - 47s 16ms/step - loss: 4.8622e-04 - mean_absolute_error: 0.0116
Epoch 2/30
2839/2839 [==============================] - 46s 16ms/step - loss: 1.8034e-04 - mean_absolute_error: 0.0080
Epoch 3/30
2839/2839 [==============================] - 45s 16ms/step - loss: 1.6576e-04 - mean_absolute_error: 0.0076
Epoch 4/30
2839/2839 [==============================] - 46s 16ms/step - loss: 1.4660e-04 - mean_absolute_error: 0.0071
Epoch 5/30
2839/2839 [==============================] - 45s 16ms/step - loss: 1.2880e-04 - mean_absolute_error: 0.0061
Epoch 6/30
2839/2839 [==============================] - 46s 16ms/step - loss: 1.1063e-04 - mean_absolute_error: 0.0059
Epoch 7/30
2839/2839 [==============================] - 46s 16ms/step - loss: 1.1188e-04 - mean_absolute_error: 0.0055
Epoch 8/30
2839/2839 [==============================] - 46s 16ms/step - loss: 9.4581e-05 - mean_absolute_error: 0.0055
Epoch 9/30
2839/2839 [==============================] - 45s 16ms/step - loss: 9.0817e-05 - mean_absolute_error: 0.0054
Epoch 10/30
2839/2839 [==============================] - 45s 16ms/step - loss: 9.3794e-05 - mean_absolute_error: 0.0053
Epoch 11/30
2839/2839 [==============================] - 45s 16ms/step - loss: 8.6526e-05 - mean_absolute_error: 0.0053
Epoch 12/30
2839/2839 [==============================] - 46s 16ms/step - loss: 9.2342e-05 - mean_absolute_error: 0.0052
Epoch 13/30
2839/2839 [==============================] - 46s 16ms/step - loss: 8.3054e-05 - mean_absolute_error: 0.0050
Epoch 14/30
2839/2839 [==============================] - 46s 16ms/step - loss: 8.5858e-05 - mean_absolute_error: 0.0051
Epoch 15/30
2839/2839 [==============================] - 46s 16ms/step - loss: 8.4113e-05 - mean_absolute_error: 0.0050
Epoch 16/30
2839/2839 [==============================] - 45s 16ms/step - loss: 7.0726e-05 - mean_absolute_error: 0.0048
Epoch 17/30
2839/2839 [==============================] - 47s 17ms/step - loss: 8.9430e-05 - mean_absolute_error: 0.0052
Epoch 18/30
2839/2839 [==============================] - 46s 16ms/step - loss: 6.3726e-05 - mean_absolute_error: 0.0045
Epoch 19/30
2839/2839 [==============================] - 46s 16ms/step - loss: 7.0662e-05 - mean_absolute_error: 0.0047
Epoch 20/30
2839/2839 [==============================] - 46s 16ms/step - loss: 7.7083e-05 - mean_absolute_error: 0.0050
Epoch 21/30
2839/2839 [==============================] - 51s 18ms/step - loss: 7.2864e-05 - mean_absolute_error: 0.0049
Epoch 22/30
2839/2839 [==============================] - 46s 16ms/step - loss: 7.3441e-05 - mean_absolute_error: 0.0047
Epoch 23/30
2839/2839 [==============================] - 46s 16ms/step - loss: 6.5800e-05 - mean_absolute_error: 0.0045
Epoch 24/30
2839/2839 [==============================] - 44s 16ms/step - loss: 6.8049e-05 - mean_absolute_error: 0.0046
Epoch 25/30
2839/2839 [==============================] - 44s 16ms/step - loss: 9.1921e-05 - mean_absolute_error: 0.0048
Epoch 26/30
2839/2839 [==============================] - 47s 16ms/step - loss: 6.0381e-05 - mean_absolute_error: 0.0044
Epoch 27/30
2839/2839 [==============================] - 45s 16ms/step - loss: 7.3052e-05 - mean_absolute_error: 0.0048
Epoch 28/30
2839/2839 [==============================] - 45s 16ms/step - loss: 6.6464e-05 - mean_absolute_error: 0.0044
Epoch 29/30
2839/2839 [==============================] - 46s 16ms/step - loss: 6.7557e-05 - mean_absolute_error: 0.0046
Epoch 30/30
2839/2839 [==============================] - 45s 16ms/step - loss: 1.5068e-04 - mean_absolute_error: 0.0046

# Let's plot the mean_squared_error and mean_absolute_error using model history on the entire dataset
plot_loss_and_mae(
    model_name="model_64_1_30_6_months_overfit",
    model_history=history_64_1_30_6_months_overfit,
)

Observations

Using six months of data as a window to predict the stock price, the mean absolute error is fluctuating i.e increasing or decreasing, the observed mean absolute error is almost similar compared to the one LSTM layer with 8 neurons. One option to potentially decrease it further is to enhance the model's complexity by adding or removing more layers or neurons.

• 1 LSTM(64) 1 Dense(1) - 6 months
• Epochs: 30
• Batch Size: 1
• Training Loss: 1.5068e-04
• Mean Absolute Error: 0.0046

Model Building with one LSTM layer and Dense layer (128-1) and 30 epochs - 3 months window

# Let's create a keras sequential model
model_128_1_30_3_months_overfit = Sequential()

# Let's add LSTM layer to the model network
model_128_1_30_3_months_overfit.add(
    LSTM(128, return_sequences=False, input_shape=(x_train_3_months.shape[1], 1))
)
# Let's add dense layer to the model network
model_128_1_30_3_months_overfit.add(Dense(1))

# Let's build the model using mean_squared_error as the loss function
# and mean_absolute_error as the evaluation metric during the compilation process
model_128_1_30_3_months_overfit.compile(
    optimizer="adam", loss="mean_squared_error", metrics=["mean_absolute_error"]
)

# Let's fit the sequential model with input features and output label
history_128_1_30_3_months_overfit = model_128_1_30_3_months_overfit.fit(
    overall_x_train_3_months, overall_y_train_3_months, batch_size=1, epochs=30
)

Epoch 1/30
2929/2929 [==============================] - 36s 12ms/step - loss: 7.0601e-04 - mean_absolute_error: 0.0127
Epoch 2/30
2929/2929 [==============================] - 34s 12ms/step - loss: 2.4975e-04 - mean_absolute_error: 0.0086
Epoch 3/30
2929/2929 [==============================] - 35s 12ms/step - loss: 1.9843e-04 - mean_absolute_error: 0.0072
Epoch 4/30
2929/2929 [==============================] - 34s 12ms/step - loss: 1.3483e-04 - mean_absolute_error: 0.0063
Epoch 5/30
2929/2929 [==============================] - 34s 12ms/step - loss: 1.1947e-04 - mean_absolute_error: 0.0063
Epoch 6/30
2929/2929 [==============================] - 34s 12ms/step - loss: 1.1321e-04 - mean_absolute_error: 0.0059
Epoch 7/30
2929/2929 [==============================] - 34s 12ms/step - loss: 1.3068e-04 - mean_absolute_error: 0.0062
Epoch 8/30
2929/2929 [==============================] - 34s 12ms/step - loss: 9.6143e-05 - mean_absolute_error: 0.0055
Epoch 9/30
2929/2929 [==============================] - 35s 12ms/step - loss: 1.2495e-04 - mean_absolute_error: 0.0057
Epoch 10/30
2929/2929 [==============================] - 36s 12ms/step - loss: 7.8260e-05 - mean_absolute_error: 0.0049
Epoch 11/30
2929/2929 [==============================] - 36s 12ms/step - loss: 1.1260e-04 - mean_absolute_error: 0.0057
Epoch 12/30
2929/2929 [==============================] - 34s 12ms/step - loss: 7.7456e-05 - mean_absolute_error: 0.0049
Epoch 13/30
2929/2929 [==============================] - 33s 11ms/step - loss: 7.9217e-05 - mean_absolute_error: 0.0048
Epoch 14/30
2929/2929 [==============================] - 34s 11ms/step - loss: 8.7932e-05 - mean_absolute_error: 0.0050
Epoch 15/30
2929/2929 [==============================] - 33s 11ms/step - loss: 8.3469e-05 - mean_absolute_error: 0.0049
Epoch 16/30
2929/2929 [==============================] - 33s 11ms/step - loss: 7.2678e-05 - mean_absolute_error: 0.0048
Epoch 17/30
2929/2929 [==============================] - 33s 11ms/step - loss: 7.4721e-05 - mean_absolute_error: 0.0047
Epoch 18/30
2929/2929 [==============================] - 34s 12ms/step - loss: 8.1947e-05 - mean_absolute_error: 0.0047
Epoch 19/30
2929/2929 [==============================] - 34s 12ms/step - loss: 6.9318e-05 - mean_absolute_error: 0.0044
Epoch 20/30
2929/2929 [==============================] - 34s 12ms/step - loss: 7.3531e-05 - mean_absolute_error: 0.0048
Epoch 21/30
2929/2929 [==============================] - 33s 11ms/step - loss: 6.6746e-05 - mean_absolute_error: 0.0045
Epoch 22/30
2929/2929 [==============================] - 34s 11ms/step - loss: 7.5705e-05 - mean_absolute_error: 0.0046
Epoch 23/30
2929/2929 [==============================] - 34s 12ms/step - loss: 7.0502e-05 - mean_absolute_error: 0.0046
Epoch 24/30
2929/2929 [==============================] - 33s 11ms/step - loss: 6.8551e-05 - mean_absolute_error: 0.0044
Epoch 25/30
2929/2929 [==============================] - 34s 11ms/step - loss: 7.0304e-05 - mean_absolute_error: 0.0044
Epoch 26/30
2929/2929 [==============================] - 34s 12ms/step - loss: 6.5197e-05 - mean_absolute_error: 0.0044
Epoch 27/30
2929/2929 [==============================] - 33s 11ms/step - loss: 6.6761e-05 - mean_absolute_error: 0.0045
Epoch 28/30
2929/2929 [==============================] - 34s 11ms/step - loss: 7.0016e-05 - mean_absolute_error: 0.0043
Epoch 29/30
2929/2929 [==============================] - 34s 12ms/step - loss: 6.8344e-05 - mean_absolute_error: 0.0044
Epoch 30/30
2929/2929 [==============================] - 34s 11ms/step - loss: 6.3571e-05 - mean_absolute_error: 0.0043

# Let's plot the mean_squared_error and mean_absolute_error using model history on the entire dataset
plot_loss_and_mae(
    model_name="model_128_1_30_3_months_overfit",
    model_history=history_128_1_30_3_months_overfit,
)

Observations

Using three months of data as a window to predict the stock price, the mean absolute error is fluctuating, i.e., increasing and decreasing gradually, the observed mean absolute error is exactly similar compared to that of one LSTM layer with 8 neurons. One option to potentially decrease it further is to increase/deccrease the number of layers/neurons.

• 1 LSTM(128) 1 Dense(1) - 3 months
• Epochs: 30
• Batch Size: 1
• Training Loss: 6.3571e-05
• Mean Absolute Error: 0.0043

Model Building with one LSTM layer and Dense layer (128-1) and 30 epochs - 6 months window

# Let's create a keras sequential model
model_128_1_30_6_months_overfit = Sequential()

# Let's add LSTM layer to the model network
model_128_1_30_6_months_overfit.add(
    LSTM(128, return_sequences=False, input_shape=(x_train_6_months.shape[1], 1))
)
# Let's add dense layer to the model network
model_128_1_30_6_months_overfit.add(Dense(1))

# Let's build the model using mean_squared_error as the loss function
# and mean_absolute_error as the evaluation metric during the compilation process
model_128_1_30_6_months_overfit.compile(
    optimizer="adam", loss="mean_squared_error", metrics=["mean_absolute_error"]
)

# Let's fit the sequential model with input features and output label
history_128_1_30_6_months_overfit = model_128_1_30_6_months_overfit.fit(
    overall_x_train_6_months, overall_y_train_6_months, batch_size=1, epochs=30
)

Epoch 1/30
2839/2839 [==============================] - 52s 18ms/step - loss: 8.2755e-04 - mean_absolute_error: 0.0121
Epoch 2/30
2839/2839 [==============================] - 39s 14ms/step - loss: 2.3723e-04 - mean_absolute_error: 0.0088
Epoch 3/30
2839/2839 [==============================] - 39s 14ms/step - loss: 1.7473e-04 - mean_absolute_error: 0.0077
Epoch 4/30
2839/2839 [==============================] - 38s 14ms/step - loss: 1.8289e-04 - mean_absolute_error: 0.0072
Epoch 5/30
2839/2839 [==============================] - 38s 13ms/step - loss: 1.2653e-04 - mean_absolute_error: 0.0063
Epoch 6/30
2839/2839 [==============================] - 38s 13ms/step - loss: 1.1183e-04 - mean_absolute_error: 0.0059
Epoch 7/30
2839/2839 [==============================] - 38s 13ms/step - loss: 1.1199e-04 - mean_absolute_error: 0.0058
Epoch 8/30
2839/2839 [==============================] - 39s 14ms/step - loss: 9.5425e-05 - mean_absolute_error: 0.0054
Epoch 9/30
2839/2839 [==============================] - 38s 13ms/step - loss: 9.6468e-05 - mean_absolute_error: 0.0057
Epoch 10/30
2839/2839 [==============================] - 38s 13ms/step - loss: 1.0635e-04 - mean_absolute_error: 0.0055
Epoch 11/30
2839/2839 [==============================] - 39s 14ms/step - loss: 9.5956e-05 - mean_absolute_error: 0.0053
Epoch 12/30
2839/2839 [==============================] - 38s 14ms/step - loss: 9.8680e-05 - mean_absolute_error: 0.0054
Epoch 13/30
2839/2839 [==============================] - 38s 13ms/step - loss: 7.6690e-05 - mean_absolute_error: 0.0049
Epoch 14/30
2839/2839 [==============================] - 39s 14ms/step - loss: 7.8743e-05 - mean_absolute_error: 0.0050
Epoch 15/30
2839/2839 [==============================] - 38s 13ms/step - loss: 8.9107e-05 - mean_absolute_error: 0.0052
Epoch 16/30
2839/2839 [==============================] - 38s 14ms/step - loss: 8.0605e-05 - mean_absolute_error: 0.0049
Epoch 17/30
2839/2839 [==============================] - 38s 13ms/step - loss: 8.0338e-05 - mean_absolute_error: 0.0049
Epoch 18/30
2839/2839 [==============================] - 39s 14ms/step - loss: 7.8170e-05 - mean_absolute_error: 0.0049
Epoch 19/30
2839/2839 [==============================] - 38s 13ms/step - loss: 8.5261e-05 - mean_absolute_error: 0.0050
Epoch 20/30
2839/2839 [==============================] - 38s 13ms/step - loss: 7.1773e-05 - mean_absolute_error: 0.0047
Epoch 21/30
2839/2839 [==============================] - 38s 13ms/step - loss: 7.1309e-05 - mean_absolute_error: 0.0047
Epoch 22/30
2839/2839 [==============================] - 39s 14ms/step - loss: 7.5829e-05 - mean_absolute_error: 0.0049
Epoch 23/30
2839/2839 [==============================] - 38s 13ms/step - loss: 6.9320e-05 - mean_absolute_error: 0.0046
Epoch 24/30
2839/2839 [==============================] - 38s 14ms/step - loss: 6.8967e-05 - mean_absolute_error: 0.0046
Epoch 25/30
2839/2839 [==============================] - 38s 14ms/step - loss: 6.5909e-05 - mean_absolute_error: 0.0046
Epoch 26/30
2839/2839 [==============================] - 38s 14ms/step - loss: 6.6570e-05 - mean_absolute_error: 0.0045
Epoch 27/30
2839/2839 [==============================] - 38s 14ms/step - loss: 6.4222e-05 - mean_absolute_error: 0.0044
Epoch 28/30
2839/2839 [==============================] - 39s 14ms/step - loss: 7.2023e-05 - mean_absolute_error: 0.0046
Epoch 29/30
2839/2839 [==============================] - 38s 14ms/step - loss: 6.4498e-05 - mean_absolute_error: 0.0044
Epoch 30/30
2839/2839 [==============================] - 39s 14ms/step - loss: 6.9634e-05 - mean_absolute_error: 0.0045

# Let's plot the mean_squared_error and mean_absolute_error using model history on the entire dataset
plot_loss_and_mae(
    model_name="model_128_1_30_6_months_overfit",
    model_history=history_128_1_30_6_months_overfit,
)

Observations

Using six months of data as a window to predict the stock price, the mean absolute error is decreasing gradually. After 10 epochs, the observed mean absolute error is almost similar to that of a single LSTM layer with 8 neurons. One option to potentially decrease it further is to increase or decrease the number of layers or neurons.

• 1 LSTM(128) 1 Dense(1) - 6 months
• Epochs: 10
• Batch Size: 1
• Training Loss: 1.0197e-04
• Mean Absolute Error: 0.0057

Model Building with 2 LSTM layers and 2 Dense layers (128-64-8-1) and 30 epochs - 3 months

# Let's create a keras sequential model
model_128_64_8_1_30_3_months_overfit = Sequential()

# Let's add two LSTM layer's to the model network
model_128_64_8_1_30_3_months_overfit.add(
    LSTM(128, return_sequences=True, input_shape=(x_train_3_months.shape[1], 1))
)
model_128_64_8_1_30_3_months_overfit.add(LSTM(64, return_sequences=False))

# Let's add dense layer to the model network
model_128_64_8_1_30_3_months_overfit.add(Dense(8))
model_128_64_8_1_30_3_months_overfit.add(Dense(1))

# Let's build the model using mean_squared_error as the loss function
# and mean_absolute_error as the evaluation metric during the compilation process
model_128_64_8_1_30_3_months_overfit.compile(
    optimizer="adam", loss="mean_squared_error", metrics=["mean_absolute_error"]
)

# Let's fit the sequential model with input features and output label
history_128_64_8_1_30_3_months_overfit = model_128_64_8_1_30_3_months_overfit.fit(
    overall_x_train_3_months, overall_y_train_3_months, batch_size=1, epochs=30
)

Epoch 1/30
2929/2929 [==============================] - 43s 14ms/step - loss: 0.0014 - mean_absolute_error: 0.0160
Epoch 2/30
2929/2929 [==============================] - 42s 14ms/step - loss: 4.5242e-04 - mean_absolute_error: 0.0114
Epoch 3/30
2929/2929 [==============================] - 42s 14ms/step - loss: 3.6377e-04 - mean_absolute_error: 0.0090
Epoch 4/30
2929/2929 [==============================] - 41s 14ms/step - loss: 4.3048e-04 - mean_absolute_error: 0.0090
Epoch 5/30
2929/2929 [==============================] - 42s 14ms/step - loss: 1.5984e-04 - mean_absolute_error: 0.0065
Epoch 6/30
2929/2929 [==============================] - 42s 14ms/step - loss: 2.5367e-04 - mean_absolute_error: 0.0080
Epoch 7/30
2929/2929 [==============================] - 43s 15ms/step - loss: 1.6424e-04 - mean_absolute_error: 0.0071
Epoch 8/30
2929/2929 [==============================] - 42s 14ms/step - loss: 1.7357e-04 - mean_absolute_error: 0.0073
Epoch 9/30
2929/2929 [==============================] - 42s 14ms/step - loss: 1.3975e-04 - mean_absolute_error: 0.0067
Epoch 10/30
2929/2929 [==============================] - 42s 14ms/step - loss: 1.4486e-04 - mean_absolute_error: 0.0072
Epoch 11/30
2929/2929 [==============================] - 42s 15ms/step - loss: 1.0134e-04 - mean_absolute_error: 0.0059
Epoch 12/30
2929/2929 [==============================] - 42s 14ms/step - loss: 1.3059e-04 - mean_absolute_error: 0.0065
Epoch 13/30
2929/2929 [==============================] - 42s 14ms/step - loss: 1.2840e-04 - mean_absolute_error: 0.0064
Epoch 14/30
2929/2929 [==============================] - 42s 14ms/step - loss: 1.3335e-04 - mean_absolute_error: 0.0063
Epoch 15/30
2929/2929 [==============================] - 42s 14ms/step - loss: 1.1542e-04 - mean_absolute_error: 0.0062
Epoch 16/30
2929/2929 [==============================] - 42s 14ms/step - loss: 1.0181e-04 - mean_absolute_error: 0.0062
Epoch 17/30
2929/2929 [==============================] - 42s 14ms/step - loss: 1.3210e-04 - mean_absolute_error: 0.0065
Epoch 18/30
2929/2929 [==============================] - 43s 15ms/step - loss: 1.2838e-04 - mean_absolute_error: 0.0064
Epoch 19/30
2929/2929 [==============================] - 43s 15ms/step - loss: 1.0413e-04 - mean_absolute_error: 0.0057
Epoch 20/30
2929/2929 [==============================] - 43s 15ms/step - loss: 1.0336e-04 - mean_absolute_error: 0.0061
Epoch 21/30
2929/2929 [==============================] - 43s 15ms/step - loss: 1.0527e-04 - mean_absolute_error: 0.0060
Epoch 22/30
2929/2929 [==============================] - 43s 15ms/step - loss: 1.2020e-04 - mean_absolute_error: 0.0060
Epoch 23/30
2929/2929 [==============================] - 43s 15ms/step - loss: 3.9251e-04 - mean_absolute_error: 0.0070
Epoch 24/30
2929/2929 [==============================] - 43s 15ms/step - loss: 7.2739e-05 - mean_absolute_error: 0.0050
Epoch 25/30
2929/2929 [==============================] - 43s 15ms/step - loss: 1.0165e-04 - mean_absolute_error: 0.0060
Epoch 26/30
2929/2929 [==============================] - 43s 15ms/step - loss: 1.1621e-04 - mean_absolute_error: 0.0062
Epoch 27/30
2929/2929 [==============================] - 43s 15ms/step - loss: 9.7587e-05 - mean_absolute_error: 0.0060
Epoch 28/30
2929/2929 [==============================] - 43s 15ms/step - loss: 8.8884e-05 - mean_absolute_error: 0.0055
Epoch 29/30
2929/2929 [==============================] - 43s 15ms/step - loss: 9.2242e-05 - mean_absolute_error: 0.0060
Epoch 30/30
2929/2929 [==============================] - 43s 15ms/step - loss: 9.9356e-05 - mean_absolute_error: 0.0059

# Let's plot the mean_squared_error and mean_absolute_error using model history on the entire dataset
plot_loss_and_mae(
    model_name="model_128_64_8_1_30_3_months_overfit",
    model_history=history_128_64_8_1_30_3_months_overfit,
)

Observations

Using three months of data as a window to predict the stock price, the training loss observed a spike between 20 and 25 epochs; however, the mean absolute error is fluctuating, i.e., increasing and decreasing. The observed mean absolute error is slightly higher than what was observed during the training of a single LSTM layer with 8 neurons. One option to potentially decrease it further is to increase or decrease the number of layers or neurons.

• 2 LSTM(128-64) 2 Dense(8-1) - 3 months
• Epochs: 30
• Batch Size: 1
• Training Loss: 9.9356e-05
• Mean Absolute Error: 0.0059

Model Building with 2 LSTM layers and 2 Dense layers (128-64-8-1) and 30 epochs - 6 months

# Let's create a keras sequential model
model_128_64_8_1_30_6_months_overfit = Sequential()

# Let's add two LSTM layer's to the model network
model_128_64_8_1_30_6_months_overfit.add(
    LSTM(128, return_sequences=True, input_shape=(x_train_6_months.shape[1], 1))
)
model_128_64_8_1_30_6_months_overfit.add(LSTM(64, return_sequences=False))

# Let's add dense layer to the model network
model_128_64_8_1_30_6_months_overfit.add(Dense(8))
model_128_64_8_1_30_6_months_overfit.add(Dense(1))

# Let's build the model using mean_squared_error as the loss function
# and mean_absolute_error as the evaluation metric during the compilation process
model_128_64_8_1_30_6_months_overfit.compile(
    optimizer="adam", loss="mean_squared_error", metrics=["mean_absolute_error"]
)

# Let's fit the sequential model with input features and output label
history_128_64_8_1_30_6_months_overfit = model_128_64_8_1_30_6_months_overfit.fit(
    overall_x_train_6_months, overall_y_train_6_months, batch_size=1, epochs=30
)

Epoch 1/30
2839/2839 [==============================] - 79s 27ms/step - loss: 0.0012 - mean_absolute_error: 0.0161
Epoch 2/30
2839/2839 [==============================] - 78s 27ms/step - loss: 5.4512e-04 - mean_absolute_error: 0.0122
Epoch 3/30
2839/2839 [==============================] - 79s 28ms/step - loss: 3.3772e-04 - mean_absolute_error: 0.0101
Epoch 4/30
2839/2839 [==============================] - 80s 28ms/step - loss: 2.9478e-04 - mean_absolute_error: 0.0085
Epoch 5/30
2839/2839 [==============================] - 81s 28ms/step - loss: 3.5223e-04 - mean_absolute_error: 0.0089
Epoch 6/30
2839/2839 [==============================] - 82s 29ms/step - loss: 2.3264e-04 - mean_absolute_error: 0.0078
Epoch 7/30
2839/2839 [==============================] - 82s 29ms/step - loss: 1.6574e-04 - mean_absolute_error: 0.0074
Epoch 8/30
2839/2839 [==============================] - 81s 28ms/step - loss: 1.6986e-04 - mean_absolute_error: 0.0074
Epoch 9/30
2839/2839 [==============================] - 81s 28ms/step - loss: 1.7507e-04 - mean_absolute_error: 0.0074
Epoch 10/30
2839/2839 [==============================] - 80s 28ms/step - loss: 1.3651e-04 - mean_absolute_error: 0.0069
Epoch 11/30
2839/2839 [==============================] - 80s 28ms/step - loss: 1.2998e-04 - mean_absolute_error: 0.0065
Epoch 12/30
2839/2839 [==============================] - 80s 28ms/step - loss: 1.3439e-04 - mean_absolute_error: 0.0070
Epoch 13/30
2839/2839 [==============================] - 80s 28ms/step - loss: 1.3219e-04 - mean_absolute_error: 0.0067
Epoch 14/30
2839/2839 [==============================] - 80s 28ms/step - loss: 1.0505e-04 - mean_absolute_error: 0.0064
Epoch 15/30
2839/2839 [==============================] - 80s 28ms/step - loss: 1.1282e-04 - mean_absolute_error: 0.0066
Epoch 16/30
2839/2839 [==============================] - 80s 28ms/step - loss: 1.2123e-04 - mean_absolute_error: 0.0064
Epoch 17/30
2839/2839 [==============================] - 80s 28ms/step - loss: 9.8212e-05 - mean_absolute_error: 0.0062
Epoch 18/30
2839/2839 [==============================] - 81s 28ms/step - loss: 1.1105e-04 - mean_absolute_error: 0.0062
Epoch 19/30
2839/2839 [==============================] - 81s 28ms/step - loss: 1.3146e-04 - mean_absolute_error: 0.0069
Epoch 20/30
2839/2839 [==============================] - 80s 28ms/step - loss: 1.2040e-04 - mean_absolute_error: 0.0065
Epoch 21/30
2839/2839 [==============================] - 80s 28ms/step - loss: 9.7492e-05 - mean_absolute_error: 0.0058
Epoch 22/30
2839/2839 [==============================] - 82s 29ms/step - loss: 1.1475e-04 - mean_absolute_error: 0.0062
Epoch 23/30
2839/2839 [==============================] - 81s 29ms/step - loss: 1.0795e-04 - mean_absolute_error: 0.0060
Epoch 24/30
2839/2839 [==============================] - 82s 29ms/step - loss: 1.0503e-04 - mean_absolute_error: 0.0062
Epoch 25/30
2839/2839 [==============================] - 80s 28ms/step - loss: 9.4084e-05 - mean_absolute_error: 0.0059
Epoch 26/30
2839/2839 [==============================] - 79s 28ms/step - loss: 1.1561e-04 - mean_absolute_error: 0.0062
Epoch 27/30
2839/2839 [==============================] - 81s 28ms/step - loss: 1.0093e-04 - mean_absolute_error: 0.0061
Epoch 28/30
2839/2839 [==============================] - 82s 29ms/step - loss: 9.4759e-05 - mean_absolute_error: 0.0057
Epoch 29/30
2839/2839 [==============================] - 83s 29ms/step - loss: 9.6674e-05 - mean_absolute_error: 0.0060
Epoch 30/30
2839/2839 [==============================] - 82s 29ms/step - loss: 9.7770e-05 - mean_absolute_error: 0.0057

# Let's plot the mean_squared_error and mean_absolute_error using model history on the entire dataset
plot_loss_and_mae(
    model_name="model_128_64_8_1_30_6_months_overfit",
    model_history=history_128_64_8_1_30_6_months_overfit,
)

Observations

Using six months of data as a window to predict the stock price, the training is decreasing gradually; however, the mean absolute error is fluctuating, i.e., increasing and decreasing. The observed mean absolute error is slightly higher than what was observed during the training of a single LSTM layer with 8 neurons. One option to potentially decrease it further is to increase/decrease the number of layers or neurons.

• 2 LSTM(128-64) 2 Dense(8-1) - 6 months
• Epochs: 30
• Batch Size: 1
• Training Loss: 9.7770e-05
• Mean Absolute Error: 0.0057

Model Building with 3 LSTM and 3 Dense layers (128-64-64-8-8-1) and 30 epochs - 3 months

# Let's create a keras sequential model
model_128_64_64_8_8_1_30_3_months_overfit = Sequential()

# Let's add two LSTM layer's to the model network
model_128_64_64_8_8_1_30_3_months_overfit.add(
    LSTM(128, return_sequences=False, input_shape=(x_train_3_months.shape[1], 1))
)
model_128_64_64_8_8_1_30_3_months_overfit.add(LSTM(64, return_sequences=True))
model_128_64_64_8_8_1_30_3_months_overfit.add(LSTM(64, return_sequences=False))

# Let's add dense layer to the model network
model_128_64_64_8_8_1_30_3_months_overfit.add(Dense(8))
model_128_64_64_8_8_1_30_3_months_overfit.add(Dense(8))
model_128_64_64_8_8_1_30_3_months_overfit.add(Dense(1))

# Let's build the model using mean_squared_error as the loss function
# and mean_absolute_error as the evaluation metric during the compilation process
model_128_64_64_8_8_1_30_3_months_overfit.compile(
    optimizer="adam", loss="mean_squared_error", metrics=["mean_absolute_error"]
)

# Let's fit the sequential model with input features and output label
history_128_64_64_8_8_1_30_3_months_overfit = (
    model_128_64_64_8_8_1_30_3_months_overfit.fit(
        overall_x_train_3_months, overall_y_train_3_months, batch_size=1, epochs=30
    )
)

Epoch 1/30
2929/2929 [==============================] - 23s 7ms/step - loss: 6.0830e-04 - mean_absolute_error: 0.0131
Epoch 2/30
2929/2929 [==============================] - 22s 7ms/step - loss: 2.3212e-04 - mean_absolute_error: 0.0085
Epoch 3/30
2929/2929 [==============================] - 23s 8ms/step - loss: 1.5396e-04 - mean_absolute_error: 0.0072
Epoch 4/30
2929/2929 [==============================] - 23s 8ms/step - loss: 1.8716e-04 - mean_absolute_error: 0.0070
Epoch 5/30
2929/2929 [==============================] - 22s 7ms/step - loss: 1.5713e-04 - mean_absolute_error: 0.0065
Epoch 6/30
2929/2929 [==============================] - 21s 7ms/step - loss: 1.1781e-04 - mean_absolute_error: 0.0057
Epoch 7/30
2929/2929 [==============================] - 22s 8ms/step - loss: 1.0321e-04 - mean_absolute_error: 0.0059
Epoch 8/30
2929/2929 [==============================] - 22s 7ms/step - loss: 1.0808e-04 - mean_absolute_error: 0.0058
Epoch 9/30
2929/2929 [==============================] - 22s 8ms/step - loss: 1.0679e-04 - mean_absolute_error: 0.0055
Epoch 10/30
2929/2929 [==============================] - 23s 8ms/step - loss: 9.2497e-05 - mean_absolute_error: 0.0051
Epoch 11/30
2929/2929 [==============================] - 23s 8ms/step - loss: 7.6332e-05 - mean_absolute_error: 0.0049
Epoch 12/30
2929/2929 [==============================] - 21s 7ms/step - loss: 8.5269e-05 - mean_absolute_error: 0.0050
Epoch 13/30
2929/2929 [==============================] - 20s 7ms/step - loss: 9.0152e-05 - mean_absolute_error: 0.0051
Epoch 14/30
2929/2929 [==============================] - 21s 7ms/step - loss: 8.6599e-05 - mean_absolute_error: 0.0052
Epoch 15/30
2929/2929 [==============================] - 21s 7ms/step - loss: 7.5812e-05 - mean_absolute_error: 0.0049
Epoch 16/30
2929/2929 [==============================] - 21s 7ms/step - loss: 7.9607e-05 - mean_absolute_error: 0.0050
Epoch 17/30
2929/2929 [==============================] - 20s 7ms/step - loss: 8.6057e-05 - mean_absolute_error: 0.0051
Epoch 18/30
2929/2929 [==============================] - 22s 7ms/step - loss: 7.4376e-05 - mean_absolute_error: 0.0048
Epoch 19/30
2929/2929 [==============================] - 22s 7ms/step - loss: 7.2214e-05 - mean_absolute_error: 0.0047
Epoch 20/30
2929/2929 [==============================] - 22s 8ms/step - loss: 7.6250e-05 - mean_absolute_error: 0.0047
Epoch 21/30
2929/2929 [==============================] - 23s 8ms/step - loss: 7.1422e-05 - mean_absolute_error: 0.0046
Epoch 22/30
2929/2929 [==============================] - 22s 8ms/step - loss: 6.6345e-05 - mean_absolute_error: 0.0045
Epoch 23/30
2929/2929 [==============================] - 23s 8ms/step - loss: 7.5610e-05 - mean_absolute_error: 0.0045
Epoch 24/30
2929/2929 [==============================] - 21s 7ms/step - loss: 6.9496e-05 - mean_absolute_error: 0.0046
Epoch 25/30
2929/2929 [==============================] - 21s 7ms/step - loss: 1.2425e-04 - mean_absolute_error: 0.0046
Epoch 26/30
2929/2929 [==============================] - 23s 8ms/step - loss: 6.6378e-05 - mean_absolute_error: 0.0043
Epoch 27/30
2929/2929 [==============================] - 22s 7ms/step - loss: 6.7362e-05 - mean_absolute_error: 0.0044
Epoch 28/30
2929/2929 [==============================] - 22s 7ms/step - loss: 6.9978e-05 - mean_absolute_error: 0.0047
Epoch 29/30
2929/2929 [==============================] - 23s 8ms/step - loss: 6.5296e-05 - mean_absolute_error: 0.0044
Epoch 30/30
2929/2929 [==============================] - 21s 7ms/step - loss: 6.8383e-05 - mean_absolute_error: 0.0044

# Let's plot the mean_squared_error and mean_absolute_error using model history on the entire dataset
plot_loss_and_mae(
    model_name="model_128_64_64_8_8_1_30_3_months_overfit",
    model_history=history_128_64_64_8_8_1_30_3_months_overfit,
)

Observations

Using three months of data as a window to predict the stock price, training loss and MAE is fluctuating i.e increasing and decreasing in between 2 and 20 epochs. Increasing the number of LSTM and dense layers has resulted in a decrease in mean absolute error compared to either one LSTM layer or two LSTM layers. To potentially further reduce the mean absolute error, increasing/decreasing the number of layers/neurons may be effective.

• 3 LSTM(128-64-64) 2 Dense(8-8-1) - 3 months
• Epochs: 30
• Batch Size: 1
• Training Loss: 6.8383e-05
• Mean Absolute Error: 0.0044

Model Building with 3 LSTM and 3 Dense layers (128-64-64-8-8-1) and 30 epochs - 6 months

# Let's create a keras sequential model
model_128_64_64_8_8_1_30_6_months_overfit = Sequential()

# Let's add two LSTM layer's to the model network
model_128_64_64_8_8_1_30_6_months_overfit.add(
    LSTM(128, return_sequences=False, input_shape=(x_train_6_months.shape[1], 1))
)
model_128_64_64_8_8_1_30_6_months_overfit.add(LSTM(64, return_sequences=True))
model_128_64_64_8_8_1_30_6_months_overfit.add(LSTM(64, return_sequences=False))

# Let's add dense layer to the model network
model_128_64_64_8_8_1_30_6_months_overfit.add(Dense(8))
model_128_64_64_8_8_1_30_6_months_overfit.add(Dense(8))
model_128_64_64_8_8_1_30_6_months_overfit.add(Dense(1))

# Let's build the model using mean_squared_error as the loss function
# and mean_absolute_error as the evaluation metric during the compilation process
model_128_64_64_8_8_1_30_6_months_overfit.compile(
    optimizer="adam", loss="mean_squared_error", metrics=["mean_absolute_error"]
)

# Let's fit the sequential model with input features and output label
history_128_64_64_8_8_1_30_6_months_overfit = (
    model_128_64_64_8_8_1_30_6_months_overfit.fit(
        overall_x_train_6_months, overall_y_train_6_months, batch_size=1, epochs=30
    )
)

Epoch 1/30
2839/2839 [==============================] - 45s 15ms/step - loss: 4.3836e-04 - mean_absolute_error: 0.0119
Epoch 2/30
2839/2839 [==============================] - 39s 14ms/step - loss: 2.1257e-04 - mean_absolute_error: 0.0079
Epoch 3/30
2839/2839 [==============================] - 43s 15ms/step - loss: 1.6693e-04 - mean_absolute_error: 0.0071
Epoch 4/30
2839/2839 [==============================] - 41s 14ms/step - loss: 1.3622e-04 - mean_absolute_error: 0.0065
Epoch 5/30
2839/2839 [==============================] - 41s 14ms/step - loss: 1.5691e-04 - mean_absolute_error: 0.0065
Epoch 6/30
2839/2839 [==============================] - 40s 14ms/step - loss: 1.0279e-04 - mean_absolute_error: 0.0056
Epoch 7/30
2839/2839 [==============================] - 41s 14ms/step - loss: 1.2296e-04 - mean_absolute_error: 0.0063
Epoch 8/30
2839/2839 [==============================] - 42s 15ms/step - loss: 1.2930e-04 - mean_absolute_error: 0.0057
Epoch 9/30
2839/2839 [==============================] - 39s 14ms/step - loss: 1.0094e-04 - mean_absolute_error: 0.0053
Epoch 10/30
2839/2839 [==============================] - 40s 14ms/step - loss: 1.0067e-04 - mean_absolute_error: 0.0057
Epoch 11/30
2839/2839 [==============================] - 39s 14ms/step - loss: 8.9177e-05 - mean_absolute_error: 0.0050
Epoch 12/30
2839/2839 [==============================] - 39s 14ms/step - loss: 8.8215e-05 - mean_absolute_error: 0.0051
Epoch 13/30
2839/2839 [==============================] - 39s 14ms/step - loss: 8.3966e-05 - mean_absolute_error: 0.0052
Epoch 14/30
2839/2839 [==============================] - 38s 13ms/step - loss: 9.5794e-05 - mean_absolute_error: 0.0052
Epoch 15/30
2839/2839 [==============================] - 40s 14ms/step - loss: 9.2568e-05 - mean_absolute_error: 0.0049
Epoch 16/30
2839/2839 [==============================] - 39s 14ms/step - loss: 9.1120e-05 - mean_absolute_error: 0.0052
Epoch 17/30
2839/2839 [==============================] - 41s 15ms/step - loss: 7.5728e-05 - mean_absolute_error: 0.0047
Epoch 18/30
2839/2839 [==============================] - 40s 14ms/step - loss: 7.9426e-05 - mean_absolute_error: 0.0047
Epoch 19/30
2839/2839 [==============================] - 41s 14ms/step - loss: 7.7429e-05 - mean_absolute_error: 0.0048
Epoch 20/30
2839/2839 [==============================] - 40s 14ms/step - loss: 7.5431e-05 - mean_absolute_error: 0.0048
Epoch 21/30
2839/2839 [==============================] - 41s 15ms/step - loss: 7.4675e-05 - mean_absolute_error: 0.0048
Epoch 22/30
2839/2839 [==============================] - 42s 15ms/step - loss: 7.9324e-05 - mean_absolute_error: 0.0048
Epoch 23/30
2839/2839 [==============================] - 40s 14ms/step - loss: 6.6612e-05 - mean_absolute_error: 0.0044
Epoch 24/30
2839/2839 [==============================] - 40s 14ms/step - loss: 7.0971e-05 - mean_absolute_error: 0.0046
Epoch 25/30
2839/2839 [==============================] - 39s 14ms/step - loss: 7.1620e-05 - mean_absolute_error: 0.0044
Epoch 26/30
2839/2839 [==============================] - 40s 14ms/step - loss: 6.6343e-05 - mean_absolute_error: 0.0043
Epoch 27/30
2839/2839 [==============================] - 39s 14ms/step - loss: 7.6904e-05 - mean_absolute_error: 0.0047
Epoch 28/30
2839/2839 [==============================] - 38s 14ms/step - loss: 7.3177e-05 - mean_absolute_error: 0.0044
Epoch 29/30
2839/2839 [==============================] - 39s 14ms/step - loss: 1.2752e-04 - mean_absolute_error: 0.0047
Epoch 30/30
2839/2839 [==============================] - 38s 14ms/step - loss: 5.8841e-05 - mean_absolute_error: 0.0043

# Let's plot the mean_squared_error and mean_absolute_error using model history on the entire dataset
plot_loss_and_mae(
    model_name="model_128_64_64_8_8_1_30_6_months_overfit",
    model_history=history_128_64_64_8_8_1_30_6_months_overfit,
)

Observations

Using six months of data as a window to predict the stock price, training loss and MAE is fluctuating i.e increasing and decreasing in between 2 and 20 epochs. Increasing the number of LSTM and dense layers has not resulted in a decrease in mean absolute error compared to either one LSTM layer or two LSTM layers. To potentially further reduce the mean absolute error, increasing/decreasing the number of layers/neurons may be effective.

• 3 LSTM(128-64-64) 2 Dense(8-8-1) - 6 months
• Epochs: 30
• Batch Size: 1
• Training Loss: 1.2476e-04
• Mean Absolute Error: 0.0070

Model Building with 4 LSTM and 3 Dense layers (8-16-16-32-8-8-1) and 30 epochs - 3 months

# Let's create a keras sequential model
model_8_16_16_32_8_8_1_30_3_months_overfit = Sequential()

# Let's add two LSTM layer's to the model network
model_8_16_16_32_8_8_1_30_3_months_overfit.add(
    LSTM(8, return_sequences=True, input_shape=(x_train_3_months.shape[1], 1))
)
model_8_16_16_32_8_8_1_30_3_months_overfit.add(LSTM(16, return_sequences=True))
model_8_16_16_32_8_8_1_30_3_months_overfit.add(LSTM(16, return_sequences=True))
model_8_16_16_32_8_8_1_30_3_months_overfit.add(LSTM(32, return_sequences=False))

# Let's add dense layer to the model network
model_8_16_16_32_8_8_1_30_3_months_overfit.add(Dense(8))
model_8_16_16_32_8_8_1_30_3_months_overfit.add(Dense(8))
model_8_16_16_32_8_8_1_30_3_months_overfit.add(Dense(1))

# Let's build the model using mean_squared_error as the loss function
# and mean_absolute_error as the evaluation metric during the compilation process
model_8_16_16_32_8_8_1_30_3_months_overfit.compile(
    optimizer="adam", loss="mean_squared_error", metrics=["mean_absolute_error"]
)

# Let's fit the sequential model with input features and output label
history_8_16_16_32_8_8_1_30_3_months_overfit = (
    model_8_16_16_32_8_8_1_30_3_months_overfit.fit(
        overall_x_train_3_months,
        overall_y_train_3_months,
        batch_size=1,
        epochs=30,
    )
)

Epoch 1/30
2929/2929 [==============================] - 59s 19ms/step - loss: 0.0025 - mean_absolute_error: 0.0265
Epoch 2/30
2929/2929 [==============================] - 56s 19ms/step - loss: 9.9980e-04 - mean_absolute_error: 0.0179
Epoch 3/30
2929/2929 [==============================] - 55s 19ms/step - loss: 5.7611e-04 - mean_absolute_error: 0.0151
Epoch 4/30
2929/2929 [==============================] - 55s 19ms/step - loss: 5.3420e-04 - mean_absolute_error: 0.0139
Epoch 5/30
2929/2929 [==============================] - 55s 19ms/step - loss: 4.3510e-04 - mean_absolute_error: 0.0130
Epoch 6/30
2929/2929 [==============================] - 56s 19ms/step - loss: 3.8278e-04 - mean_absolute_error: 0.0121
Epoch 7/30
2929/2929 [==============================] - 55s 19ms/step - loss: 2.8334e-04 - mean_absolute_error: 0.0108
Epoch 8/30
2929/2929 [==============================] - 56s 19ms/step - loss: 2.2956e-04 - mean_absolute_error: 0.0097
Epoch 9/30
2929/2929 [==============================] - 56s 19ms/step - loss: 2.5271e-04 - mean_absolute_error: 0.0094
Epoch 10/30
2929/2929 [==============================] - 55s 19ms/step - loss: 2.0391e-04 - mean_absolute_error: 0.0083
Epoch 11/30
2929/2929 [==============================] - 55s 19ms/step - loss: 2.1421e-04 - mean_absolute_error: 0.0084
Epoch 12/30
2929/2929 [==============================] - 55s 19ms/step - loss: 1.5674e-04 - mean_absolute_error: 0.0078
Epoch 13/30
2929/2929 [==============================] - 56s 19ms/step - loss: 1.5980e-04 - mean_absolute_error: 0.0076
Epoch 14/30
2929/2929 [==============================] - 55s 19ms/step - loss: 1.7944e-04 - mean_absolute_error: 0.0079
Epoch 15/30
2929/2929 [==============================] - 56s 19ms/step - loss: 1.6180e-04 - mean_absolute_error: 0.0075
Epoch 16/30
2929/2929 [==============================] - 55s 19ms/step - loss: 1.4239e-04 - mean_absolute_error: 0.0071
Epoch 17/30
2929/2929 [==============================] - 55s 19ms/step - loss: 1.7252e-04 - mean_absolute_error: 0.0082
Epoch 18/30
2929/2929 [==============================] - 56s 19ms/step - loss: 1.3790e-04 - mean_absolute_error: 0.0071
Epoch 19/30
2929/2929 [==============================] - 56s 19ms/step - loss: 1.1658e-04 - mean_absolute_error: 0.0069
Epoch 20/30
2929/2929 [==============================] - 55s 19ms/step - loss: 1.4159e-04 - mean_absolute_error: 0.0073
Epoch 21/30
2929/2929 [==============================] - 56s 19ms/step - loss: 1.3731e-04 - mean_absolute_error: 0.0073
Epoch 22/30
2929/2929 [==============================] - 55s 19ms/step - loss: 1.3482e-04 - mean_absolute_error: 0.0075
Epoch 23/30
2929/2929 [==============================] - 56s 19ms/step - loss: 1.2868e-04 - mean_absolute_error: 0.0072
Epoch 24/30
2929/2929 [==============================] - 56s 19ms/step - loss: 1.3530e-04 - mean_absolute_error: 0.0070
Epoch 25/30
2929/2929 [==============================] - 56s 19ms/step - loss: 1.3898e-04 - mean_absolute_error: 0.0078
Epoch 26/30
2929/2929 [==============================] - 56s 19ms/step - loss: 1.2084e-04 - mean_absolute_error: 0.0071
Epoch 27/30
2929/2929 [==============================] - 55s 19ms/step - loss: 1.4523e-04 - mean_absolute_error: 0.0074
Epoch 28/30
2929/2929 [==============================] - 56s 19ms/step - loss: 1.1535e-04 - mean_absolute_error: 0.0068
Epoch 29/30
2929/2929 [==============================] - 56s 19ms/step - loss: 1.1719e-04 - mean_absolute_error: 0.0069
Epoch 30/30
2929/2929 [==============================] - 56s 19ms/step - loss: 1.2815e-04 - mean_absolute_error: 0.0068

# Let's plot the mean_squared_error and mean_absolute_error using model history on the entire dataset
plot_loss_and_mae(
    model_name="model_8_16_16_32_8_8_1_30_3_months_overfit",
    model_history=history_8_16_16_32_8_8_1_30_3_months_overfit,
)

Observations

Using three months of data as a window to predict the stock price, training loss and mean absolute error are decreasing gradually. Increasing the number of layers from three LSTM and three dense layer to four LSTM and three dense layers has not resulted in a decrease in mean absolute error when compared to one LSTM and one dense layer, which was trained for 30 epochs. To potentially further reduce the mean absolute error, increasing or decreasing the number of layers/neurons may be effective.

• 3 LSTM(8-16-16-32) 3 Dense(8-8-1) - 3 months
• Epochs: 30
• Batch Size: 1
• Training Loss: 1.2815e-04
• Mean Absolute Error: 0.0068

Model Building with 4 LSTM and 3 Dense layers (8-16-16-32-8-8-1) and 30 epochs - 6 months

# Let's create a keras sequential model
model_8_16_16_32_8_8_1_30_6_months_overfit = Sequential()

# Let's add two LSTM layer's to the model network
model_8_16_16_32_8_8_1_30_6_months_overfit.add(
    LSTM(8, return_sequences=True, input_shape=(x_train_6_months.shape[1], 1))
)
model_8_16_16_32_8_8_1_30_6_months_overfit.add(LSTM(16, return_sequences=True))
model_8_16_16_32_8_8_1_30_6_months_overfit.add(LSTM(16, return_sequences=True))
model_8_16_16_32_8_8_1_30_6_months_overfit.add(LSTM(32, return_sequences=False))

# Let's add dense layer to the model network
model_8_16_16_32_8_8_1_30_6_months_overfit.add(Dense(8))
model_8_16_16_32_8_8_1_30_6_months_overfit.add(Dense(8))
model_8_16_16_32_8_8_1_30_6_months_overfit.add(Dense(1))

# Let's build the model using mean_squared_error as the loss function
# and mean_absolute_error as the evaluation metric during the compilation process
model_8_16_16_32_8_8_1_30_6_months_overfit.compile(
    optimizer="adam", loss="mean_squared_error", metrics=["mean_absolute_error"]
)

# Let's fit the sequential model with input features and output label
history_8_16_16_32_8_8_1_30_6_months_overfit = (
    model_8_16_16_32_8_8_1_30_6_months_overfit.fit(
        overall_x_train_6_months,
        overall_y_train_6_months,
        batch_size=1,
        epochs=30,
    )
)

Epoch 1/30
2839/2839 [==============================] - 112s 39ms/step - loss: 0.0016 - mean_absolute_error: 0.0235
Epoch 2/30
2839/2839 [==============================] - 110s 39ms/step - loss: 8.7240e-04 - mean_absolute_error: 0.0163
Epoch 3/30
2839/2839 [==============================] - 109s 38ms/step - loss: 6.1269e-04 - mean_absolute_error: 0.0146
Epoch 4/30
2839/2839 [==============================] - 109s 38ms/step - loss: 4.8074e-04 - mean_absolute_error: 0.0130
Epoch 5/30
2839/2839 [==============================] - 110s 39ms/step - loss: 3.4548e-04 - mean_absolute_error: 0.0114
Epoch 6/30
2839/2839 [==============================] - 109s 38ms/step - loss: 2.9197e-04 - mean_absolute_error: 0.0106
Epoch 7/30
2839/2839 [==============================] - 109s 38ms/step - loss: 2.6560e-04 - mean_absolute_error: 0.0101
Epoch 8/30
2839/2839 [==============================] - 110s 39ms/step - loss: 2.6036e-04 - mean_absolute_error: 0.0100
Epoch 9/30
2839/2839 [==============================] - 109s 38ms/step - loss: 2.7009e-04 - mean_absolute_error: 0.0094
Epoch 10/30
2839/2839 [==============================] - 109s 39ms/step - loss: 2.4535e-04 - mean_absolute_error: 0.0082
Epoch 11/30
2839/2839 [==============================] - 110s 39ms/step - loss: 1.6164e-04 - mean_absolute_error: 0.0076
Epoch 12/30
2839/2839 [==============================] - 109s 38ms/step - loss: 1.7594e-04 - mean_absolute_error: 0.0075
Epoch 13/30
2839/2839 [==============================] - 109s 38ms/step - loss: 1.6232e-04 - mean_absolute_error: 0.0080
Epoch 14/30
2839/2839 [==============================] - 109s 39ms/step - loss: 1.9002e-04 - mean_absolute_error: 0.0081
Epoch 15/30
2839/2839 [==============================] - 110s 39ms/step - loss: 1.5326e-04 - mean_absolute_error: 0.0077
Epoch 16/30
2839/2839 [==============================] - 110s 39ms/step - loss: 1.6059e-04 - mean_absolute_error: 0.0077
Epoch 17/30
2839/2839 [==============================] - 109s 38ms/step - loss: 1.7460e-04 - mean_absolute_error: 0.0074
Epoch 18/30
2839/2839 [==============================] - 111s 39ms/step - loss: 1.4297e-04 - mean_absolute_error: 0.0074
Epoch 19/30
2839/2839 [==============================] - 110s 39ms/step - loss: 1.8917e-04 - mean_absolute_error: 0.0081
Epoch 20/30
2839/2839 [==============================] - 109s 38ms/step - loss: 1.5843e-04 - mean_absolute_error: 0.0077
Epoch 21/30
2839/2839 [==============================] - 109s 39ms/step - loss: 1.4226e-04 - mean_absolute_error: 0.0068
Epoch 22/30
2839/2839 [==============================] - 109s 38ms/step - loss: 1.3536e-04 - mean_absolute_error: 0.0070
Epoch 23/30
2839/2839 [==============================] - 109s 38ms/step - loss: 2.5927e-04 - mean_absolute_error: 0.0075
Epoch 24/30
2839/2839 [==============================] - 109s 38ms/step - loss: 1.0815e-04 - mean_absolute_error: 0.0061
Epoch 25/30
2839/2839 [==============================] - 109s 38ms/step - loss: 1.1880e-04 - mean_absolute_error: 0.0068
Epoch 26/30
2839/2839 [==============================] - 109s 38ms/step - loss: 1.2213e-04 - mean_absolute_error: 0.0069
Epoch 27/30
2839/2839 [==============================] - 109s 38ms/step - loss: 1.2160e-04 - mean_absolute_error: 0.0073
Epoch 28/30
2839/2839 [==============================] - 109s 38ms/step - loss: 1.3934e-04 - mean_absolute_error: 0.0070
Epoch 29/30
2839/2839 [==============================] - 109s 38ms/step - loss: 1.4133e-04 - mean_absolute_error: 0.0071
Epoch 30/30
2839/2839 [==============================] - 109s 38ms/step - loss: 1.3762e-04 - mean_absolute_error: 0.0071

# Let's plot the mean_squared_error and mean_absolute_error using model history on the entire dataset
plot_loss_and_mae(
    model_name="model_8_16_16_32_8_8_1_30_6_months_overfit",
    model_history=history_8_16_16_32_8_8_1_30_6_months_overfit,
)

Observations

Using six months of data as a window to predict the stock price, training loss and mean absolute error are decreasing gradually. Increasing the number of layers from three LSTM and three dense layer to four LSTM and three dense layers has not resulted in a decrease in mean absolute error when compared to one LSTM and one dense layer, which was trained for 30 epochs. To potentially further reduce the mean absolute error, increasing or decreasing the number of layers/neurons may be effective.

• 3 LSTM(8-16-16-32) 3 Dense(8-8-1) - 6 months
• Epochs: 30
• Batch Size: 1
• Training Loss: 1.3762e-04
• Mean Absolute Error: 0.0071

Model Building with 4 LSTM and 4 Dense layers (8-16-16-32-32-16-16-1) and 30 epochs - 3 months

# Let's create a keras sequential model
model_8_16_16_32_32_16_16_1_30_3_months_overfit = Sequential()

# Let's add two LSTM layer's to the model network
model_8_16_16_32_32_16_16_1_30_3_months_overfit.add(
    LSTM(8, return_sequences=True, input_shape=(x_train_3_months.shape[1], 1))
)
model_8_16_16_32_32_16_16_1_30_3_months_overfit.add(LSTM(16, return_sequences=True))
model_8_16_16_32_32_16_16_1_30_3_months_overfit.add(LSTM(16, return_sequences=True))
model_8_16_16_32_32_16_16_1_30_3_months_overfit.add(LSTM(32, return_sequences=False))

# Let's add dense layer to the model network
model_8_16_16_32_32_16_16_1_30_3_months_overfit.add(Dense(32))
model_8_16_16_32_32_16_16_1_30_3_months_overfit.add(Dense(16))
model_8_16_16_32_32_16_16_1_30_3_months_overfit.add(Dense(16))
model_8_16_16_32_32_16_16_1_30_3_months_overfit.add(Dense(1))

# Let's build the model using mean_squared_error as the loss function
# and mean_absolute_error as the evaluation metric during the compilation process
model_8_16_16_32_32_16_16_1_30_3_months_overfit.compile(
    optimizer="adam", loss="mean_squared_error", metrics=["mean_absolute_error"]
)

# Let's fit the sequential model with input features and output label
history_8_16_16_32_32_16_16_1_30_3_months_overfit = (
    model_8_16_16_32_32_16_16_1_30_3_months_overfit.fit(
        overall_x_train_3_months,
        overall_y_train_3_months,
        batch_size=1,
        epochs=30,
    )
)

Epoch 1/30
2929/2929 [==============================] - 57s 19ms/step - loss: 0.0023 - mean_absolute_error: 0.0268
Epoch 2/30
2929/2929 [==============================] - 55s 19ms/step - loss: 0.0013 - mean_absolute_error: 0.0213
Epoch 3/30
2929/2929 [==============================] - 55s 19ms/step - loss: 0.0017 - mean_absolute_error: 0.0207
Epoch 4/30
2929/2929 [==============================] - 55s 19ms/step - loss: 6.6528e-04 - mean_absolute_error: 0.0147
Epoch 5/30
2929/2929 [==============================] - 55s 19ms/step - loss: 5.6577e-04 - mean_absolute_error: 0.0157
Epoch 6/30
2929/2929 [==============================] - 55s 19ms/step - loss: 4.7583e-04 - mean_absolute_error: 0.0138
Epoch 7/30
2929/2929 [==============================] - 56s 19ms/step - loss: 4.7474e-04 - mean_absolute_error: 0.0136
Epoch 8/30
2929/2929 [==============================] - 56s 19ms/step - loss: 5.6703e-04 - mean_absolute_error: 0.0136
Epoch 9/30
2929/2929 [==============================] - 55s 19ms/step - loss: 3.3538e-04 - mean_absolute_error: 0.0111
Epoch 10/30
2929/2929 [==============================] - 56s 19ms/step - loss: 3.5566e-04 - mean_absolute_error: 0.0117
Epoch 11/30
2929/2929 [==============================] - 55s 19ms/step - loss: 2.9405e-04 - mean_absolute_error: 0.0107
Epoch 12/30
2929/2929 [==============================] - 55s 19ms/step - loss: 1.9365e-04 - mean_absolute_error: 0.0091
Epoch 13/30
2929/2929 [==============================] - 55s 19ms/step - loss: 2.2810e-04 - mean_absolute_error: 0.0093
Epoch 14/30
2929/2929 [==============================] - 55s 19ms/step - loss: 1.9849e-04 - mean_absolute_error: 0.0088
Epoch 15/30
2929/2929 [==============================] - 56s 19ms/step - loss: 2.7580e-04 - mean_absolute_error: 0.0088
Epoch 16/30
2929/2929 [==============================] - 56s 19ms/step - loss: 1.4565e-04 - mean_absolute_error: 0.0073
Epoch 17/30
2929/2929 [==============================] - 55s 19ms/step - loss: 2.0090e-04 - mean_absolute_error: 0.0085
Epoch 18/30
2929/2929 [==============================] - 55s 19ms/step - loss: 1.8893e-04 - mean_absolute_error: 0.0080
Epoch 19/30
2929/2929 [==============================] - 55s 19ms/step - loss: 1.7670e-04 - mean_absolute_error: 0.0080
Epoch 20/30
2929/2929 [==============================] - 55s 19ms/step - loss: 1.6823e-04 - mean_absolute_error: 0.0080
Epoch 21/30
2929/2929 [==============================] - 56s 19ms/step - loss: 1.3060e-04 - mean_absolute_error: 0.0072
Epoch 22/30
2929/2929 [==============================] - 56s 19ms/step - loss: 2.2985e-04 - mean_absolute_error: 0.0078
Epoch 23/30
2929/2929 [==============================] - 56s 19ms/step - loss: 1.2910e-04 - mean_absolute_error: 0.0072
Epoch 24/30
2929/2929 [==============================] - 56s 19ms/step - loss: 1.5141e-04 - mean_absolute_error: 0.0074
Epoch 25/30
2929/2929 [==============================] - 55s 19ms/step - loss: 1.6418e-04 - mean_absolute_error: 0.0079
Epoch 26/30
2929/2929 [==============================] - 56s 19ms/step - loss: 1.3464e-04 - mean_absolute_error: 0.0074
Epoch 27/30
2929/2929 [==============================] - 56s 19ms/step - loss: 1.5492e-04 - mean_absolute_error: 0.0074
Epoch 28/30
2929/2929 [==============================] - 56s 19ms/step - loss: 1.2355e-04 - mean_absolute_error: 0.0072
Epoch 29/30
2929/2929 [==============================] - 55s 19ms/step - loss: 1.3534e-04 - mean_absolute_error: 0.0073
Epoch 30/30
2929/2929 [==============================] - 56s 19ms/step - loss: 1.1268e-04 - mean_absolute_error: 0.0067

# Let's plot the mean_squared_error and mean_absolute_error using model history on the entire dataset
plot_loss_and_mae(
    model_name="model_8_16_16_32_32_16_16_1_30_3_months_overfit",
    model_history=history_8_16_16_32_32_16_16_1_30_3_months_overfit,
)

Observations

Using three months of data as a window to predict the stock price, training loss and mean absolute error are fluctuating i.e increasing and decreasing. Increasing the number of LSTM and Dense layers and neurons has not resulted in a decrease in mean absolute error when compared to one LSTM and one dense layer, which was trained for 30 epochs. To potentially further reduce the mean absolute error, increasing/decreasing the number of LSTM layers or neurons may result in a decrease in MAE.

• 4 LSTM(8-16-16-32) 4 Dense(32-32-16-16-1) - 3 months
• Epochs: 30
• Batch Size: 1
• Training Loss: 1.1268e-04
• Mean Absolute Error: 0.0067

Model Building with 4 LSTM and 4 Dense layers (8-16-16-32-32-16-16-1) and 30 epochs - 6 months

# Let's create a keras sequential model
model_8_16_16_32_32_16_16_1_30_6_months_overfit = Sequential()

# Let's add two LSTM layer's to the model network
model_8_16_16_32_32_16_16_1_30_6_months_overfit.add(
    LSTM(8, return_sequences=True, input_shape=(x_train_6_months.shape[1], 1))
)
model_8_16_16_32_32_16_16_1_30_6_months_overfit.add(LSTM(16, return_sequences=True))
model_8_16_16_32_32_16_16_1_30_6_months_overfit.add(LSTM(16, return_sequences=True))
model_8_16_16_32_32_16_16_1_30_6_months_overfit.add(LSTM(32, return_sequences=False))

# Let's add dense layer to the model network
model_8_16_16_32_32_16_16_1_30_6_months_overfit.add(Dense(32))
model_8_16_16_32_32_16_16_1_30_6_months_overfit.add(Dense(16))
model_8_16_16_32_32_16_16_1_30_6_months_overfit.add(Dense(16))
model_8_16_16_32_32_16_16_1_30_6_months_overfit.add(Dense(1))

# Let's build the model using mean_squared_error as the loss function
# and mean_absolute_error as the evaluation metric during the compilation process
model_8_16_16_32_32_16_16_1_30_6_months_overfit.compile(
    optimizer="adam", loss="mean_squared_error", metrics=["mean_absolute_error"]
)

# Let's fit the sequential model with input features and output label
history_8_16_16_32_32_16_16_1_30_6_months_overfit = (
    model_8_16_16_32_32_16_16_1_30_6_months_overfit.fit(
        overall_x_train_6_months,
        overall_y_train_6_months,
        batch_size=1,
        epochs=30,
    )
)

Epoch 1/30
2839/2839 [==============================] - 114s 39ms/step - loss: 0.0017 - mean_absolute_error: 0.0244
Epoch 2/30
2839/2839 [==============================] - 111s 39ms/step - loss: 0.0011 - mean_absolute_error: 0.0196
Epoch 3/30
2839/2839 [==============================] - 111s 39ms/step - loss: 7.5172e-04 - mean_absolute_error: 0.0166
Epoch 4/30
2839/2839 [==============================] - 111s 39ms/step - loss: 9.1018e-04 - mean_absolute_error: 0.0164
Epoch 5/30
2839/2839 [==============================] - 111s 39ms/step - loss: 4.9226e-04 - mean_absolute_error: 0.0137
Epoch 6/30
2839/2839 [==============================] - 111s 39ms/step - loss: 4.3106e-04 - mean_absolute_error: 0.0131
Epoch 7/30
2839/2839 [==============================] - 110s 39ms/step - loss: 3.5264e-04 - mean_absolute_error: 0.0113
Epoch 8/30
2839/2839 [==============================] - 109s 38ms/step - loss: 2.5843e-04 - mean_absolute_error: 0.0106
Epoch 9/30
2839/2839 [==============================] - 110s 39ms/step - loss: 3.1711e-04 - mean_absolute_error: 0.0108
Epoch 10/30
2839/2839 [==============================] - 109s 38ms/step - loss: 2.8058e-04 - mean_absolute_error: 0.0099
Epoch 11/30
2839/2839 [==============================] - 109s 38ms/step - loss: 2.2865e-04 - mean_absolute_error: 0.0091
Epoch 12/30
2839/2839 [==============================] - 109s 39ms/step - loss: 2.7076e-04 - mean_absolute_error: 0.0092
Epoch 13/30
2839/2839 [==============================] - 109s 39ms/step - loss: 1.9770e-04 - mean_absolute_error: 0.0082
Epoch 14/30
2839/2839 [==============================] - 110s 39ms/step - loss: 1.8036e-04 - mean_absolute_error: 0.0086
Epoch 15/30
2839/2839 [==============================] - 109s 38ms/step - loss: 1.6562e-04 - mean_absolute_error: 0.0081
Epoch 16/30
2839/2839 [==============================] - 109s 38ms/step - loss: 1.8933e-04 - mean_absolute_error: 0.0083
Epoch 17/30
2839/2839 [==============================] - 110s 39ms/step - loss: 1.7006e-04 - mean_absolute_error: 0.0081
Epoch 18/30
2839/2839 [==============================] - 110s 39ms/step - loss: 1.4325e-04 - mean_absolute_error: 0.0077
Epoch 19/30
2839/2839 [==============================] - 109s 38ms/step - loss: 1.6287e-04 - mean_absolute_error: 0.0080
Epoch 20/30
2839/2839 [==============================] - 110s 39ms/step - loss: 1.6133e-04 - mean_absolute_error: 0.0078
Epoch 21/30
2839/2839 [==============================] - 109s 38ms/step - loss: 1.4493e-04 - mean_absolute_error: 0.0076
Epoch 22/30
2839/2839 [==============================] - 111s 39ms/step - loss: 1.8942e-04 - mean_absolute_error: 0.0077
Epoch 23/30
2839/2839 [==============================] - 111s 39ms/step - loss: 1.3105e-04 - mean_absolute_error: 0.0068
Epoch 24/30
2839/2839 [==============================] - 110s 39ms/step - loss: 1.3226e-04 - mean_absolute_error: 0.0074
Epoch 25/30
2839/2839 [==============================] - 110s 39ms/step - loss: 1.4728e-04 - mean_absolute_error: 0.0073
Epoch 26/30
2839/2839 [==============================] - 110s 39ms/step - loss: 1.2277e-04 - mean_absolute_error: 0.0071
Epoch 27/30
2839/2839 [==============================] - 109s 38ms/step - loss: 1.3718e-04 - mean_absolute_error: 0.0070
Epoch 28/30
2839/2839 [==============================] - 110s 39ms/step - loss: 1.1557e-04 - mean_absolute_error: 0.0069
Epoch 29/30
2839/2839 [==============================] - 110s 39ms/step - loss: 1.1990e-04 - mean_absolute_error: 0.0070
Epoch 30/30
2839/2839 [==============================] - 110s 39ms/step - loss: 1.4419e-04 - mean_absolute_error: 0.0068

# Let's plot the mean_squared_error and mean_absolute_error using model history on the entire dataset
plot_loss_and_mae(
    model_name="model_8_16_16_32_32_16_16_1_30_6_months_overfit",
    model_history=history_8_16_16_32_32_16_16_1_30_6_months_overfit,
)

Observations

Using six months of data as a window to predict the stock price, training loss and mean absolute error are decreasing gradually. Increasing the number of LSTM/Dense layers and neurons has not resulted in a decrease in mean absolute error when compared to one LSTM and one dense layer, which was trained for 30 epochs. To potentially further reduce the mean absolute error, increasing/decreasing the number of LSTM layers or neurons may result in a decrease in MAE.

• 4 LSTM(8-16-16-32) 4 Dense(32-32-16-16-1) - 6 months
• Epochs: 30
• Batch Size: 1
• Training Loss: 1.4419e-04
• Mean Absolute Error: 0.0068

Model Building with 5 LSTM and 3 Dense layers (8-16-16-32-64-8-8-1) and 30 epochs - 3 months

# Let's create a keras sequential model
model_8_16_16_32_64_8_8_1_30_3_months_overfit = Sequential()

# Let's add two LSTM layer's to the model network
model_8_16_16_32_64_8_8_1_30_3_months_overfit.add(
    LSTM(8, return_sequences=True, input_shape=(x_train_3_months.shape[1], 1))
)
model_8_16_16_32_64_8_8_1_30_3_months_overfit.add(LSTM(16, return_sequences=True))
model_8_16_16_32_64_8_8_1_30_3_months_overfit.add(LSTM(16, return_sequences=True))
model_8_16_16_32_64_8_8_1_30_3_months_overfit.add(LSTM(32, return_sequences=True))
model_8_16_16_32_64_8_8_1_30_3_months_overfit.add(LSTM(64, return_sequences=False))

# Let's add dense layer to the model network
model_8_16_16_32_64_8_8_1_30_3_months_overfit.add(Dense(8))
model_8_16_16_32_64_8_8_1_30_3_months_overfit.add(Dense(8))
model_8_16_16_32_64_8_8_1_30_3_months_overfit.add(Dense(1))

# Let's build the model using mean_squared_error as the loss function
# and mean_absolute_error as the evaluation metric during the compilation process
model_8_16_16_32_64_8_8_1_30_3_months_overfit.compile(
    optimizer="adam", loss="mean_squared_error", metrics=["mean_absolute_error"]
)

# Let's fit the sequential model with input features and output label
history_8_16_16_32_64_8_8_1_30_3_months_overfit = (
    model_8_16_16_32_64_8_8_1_30_3_months_overfit.fit(
        overall_x_train_3_months,
        overall_y_train_3_months,
        batch_size=1,
        epochs=30,
    )
)

Epoch 1/30
2929/2929 [==============================] - 75s 25ms/step - loss: 0.0033 - mean_absolute_error: 0.0295
Epoch 2/30
2929/2929 [==============================] - 73s 25ms/step - loss: 8.9989e-04 - mean_absolute_error: 0.0191
Epoch 3/30
2929/2929 [==============================] - 72s 25ms/step - loss: 8.9927e-04 - mean_absolute_error: 0.0183
Epoch 4/30
2929/2929 [==============================] - 72s 25ms/step - loss: 6.7194e-04 - mean_absolute_error: 0.0160
Epoch 5/30
2929/2929 [==============================] - 72s 25ms/step - loss: 5.3385e-04 - mean_absolute_error: 0.0151
Epoch 6/30
2929/2929 [==============================] - 73s 25ms/step - loss: 4.0701e-04 - mean_absolute_error: 0.0127
Epoch 7/30
2929/2929 [==============================] - 73s 25ms/step - loss: 3.5668e-04 - mean_absolute_error: 0.0118
Epoch 8/30
2929/2929 [==============================] - 72s 25ms/step - loss: 2.6977e-04 - mean_absolute_error: 0.0104
Epoch 9/30
2929/2929 [==============================] - 74s 25ms/step - loss: 3.0250e-04 - mean_absolute_error: 0.0100
Epoch 10/30
2929/2929 [==============================] - 73s 25ms/step - loss: 2.1383e-04 - mean_absolute_error: 0.0096
Epoch 11/30
2929/2929 [==============================] - 73s 25ms/step - loss: 2.8694e-04 - mean_absolute_error: 0.0094
Epoch 12/30
2929/2929 [==============================] - 73s 25ms/step - loss: 1.9460e-04 - mean_absolute_error: 0.0084
Epoch 13/30
2929/2929 [==============================] - 73s 25ms/step - loss: 2.0186e-04 - mean_absolute_error: 0.0087
Epoch 14/30
2929/2929 [==============================] - 72s 25ms/step - loss: 1.8550e-04 - mean_absolute_error: 0.0085
Epoch 15/30
2929/2929 [==============================] - 71s 24ms/step - loss: 1.7053e-04 - mean_absolute_error: 0.0083
Epoch 16/30
2929/2929 [==============================] - 71s 24ms/step - loss: 1.5805e-04 - mean_absolute_error: 0.0079
Epoch 17/30
2929/2929 [==============================] - 71s 24ms/step - loss: 1.9572e-04 - mean_absolute_error: 0.0086
Epoch 18/30
2929/2929 [==============================] - 71s 24ms/step - loss: 1.3474e-04 - mean_absolute_error: 0.0076
Epoch 19/30
2929/2929 [==============================] - 72s 24ms/step - loss: 1.6497e-04 - mean_absolute_error: 0.0087
Epoch 20/30
2929/2929 [==============================] - 71s 24ms/step - loss: 1.5421e-04 - mean_absolute_error: 0.0079
Epoch 21/30
2929/2929 [==============================] - 71s 24ms/step - loss: 1.4952e-04 - mean_absolute_error: 0.0077
Epoch 22/30
2929/2929 [==============================] - 71s 24ms/step - loss: 1.6481e-04 - mean_absolute_error: 0.0083
Epoch 23/30
2929/2929 [==============================] - 71s 24ms/step - loss: 1.4552e-04 - mean_absolute_error: 0.0077
Epoch 24/30
2929/2929 [==============================] - 71s 24ms/step - loss: 1.7313e-04 - mean_absolute_error: 0.0080
Epoch 25/30
2929/2929 [==============================] - 70s 24ms/step - loss: 1.3116e-04 - mean_absolute_error: 0.0073
Epoch 26/30
2929/2929 [==============================] - 71s 24ms/step - loss: 1.3943e-04 - mean_absolute_error: 0.0075
Epoch 27/30
2929/2929 [==============================] - 71s 24ms/step - loss: 1.3604e-04 - mean_absolute_error: 0.0074
Epoch 28/30
2929/2929 [==============================] - 71s 24ms/step - loss: 1.5024e-04 - mean_absolute_error: 0.0078
Epoch 29/30
2929/2929 [==============================] - 71s 24ms/step - loss: 1.5396e-04 - mean_absolute_error: 0.0078
Epoch 30/30
2929/2929 [==============================] - 71s 24ms/step - loss: 1.4442e-04 - mean_absolute_error: 0.0077

# Let's plot the mean_squared_error and mean_absolute_error using model history on the entire dataset
plot_loss_and_mae(
    model_name="model_8_16_16_32_64_8_8_1_30_3_months_overfit",
    model_history=history_8_16_16_32_64_8_8_1_30_3_months_overfit,
)

Observations

Using three months of data as a window to predict the stock price, training loss and mean absolute error are decreasing gradually. Increasing the number of LSTM layers has not resulted in a decrease in mean absolute error when compared to one LSTM and one dense layer, which was trained for 30 epochs. To potentially further reduce the mean absolute error, increasing/decreasing the number of LSTM layers or neurons may result in a decrease in MAE.

• 5 LSTM(8-16-16-32-64) 3 Dense(8-8-1) - 3 months
• Epochs: 30
• Batch Size: 1
• Training Loss: 1.4442e-04
• Mean Absolute Error: 0.0077

Model Building with 5 LSTM and 3 Dense layers (8-16-16-32-64-8-8-1) and 30 epochs - 6 months

# Let's create a keras sequential model
model_8_16_16_32_64_8_8_1_30_6_months_overfit = Sequential()

# Let's add two LSTM layer's to the model network
model_8_16_16_32_64_8_8_1_30_6_months_overfit.add(
    LSTM(8, return_sequences=True, input_shape=(x_train_6_months.shape[1], 1))
)
model_8_16_16_32_64_8_8_1_30_6_months_overfit.add(LSTM(16, return_sequences=True))
model_8_16_16_32_64_8_8_1_30_6_months_overfit.add(LSTM(16, return_sequences=True))
model_8_16_16_32_64_8_8_1_30_6_months_overfit.add(LSTM(32, return_sequences=True))
model_8_16_16_32_64_8_8_1_30_6_months_overfit.add(LSTM(64, return_sequences=False))

# Let's add dense layer to the model network
model_8_16_16_32_64_8_8_1_30_6_months_overfit.add(Dense(8))
model_8_16_16_32_64_8_8_1_30_6_months_overfit.add(Dense(8))
model_8_16_16_32_64_8_8_1_30_6_months_overfit.add(Dense(1))

# Let's build the model using mean_squared_error as the loss function
# and mean_absolute_error as the evaluation metric during the compilation process
model_8_16_16_32_64_8_8_1_30_6_months_overfit.compile(
    optimizer="adam", loss="mean_squared_error", metrics=["mean_absolute_error"]
)

# Let's fit the sequential model with input features and output label
history_8_16_16_32_64_8_8_1_30_6_months_overfit = (
    model_8_16_16_32_64_8_8_1_30_6_months_overfit.fit(
        overall_x_train_6_months,
        overall_y_train_6_months,
        batch_size=1,
        epochs=30,
    )
)

Epoch 1/30
2839/2839 [==============================] - 147s 51ms/step - loss: 0.0027 - mean_absolute_error: 0.0279
Epoch 2/30
2839/2839 [==============================] - 143s 50ms/step - loss: 0.0013 - mean_absolute_error: 0.0205
Epoch 3/30
2839/2839 [==============================] - 144s 51ms/step - loss: 6.7109e-04 - mean_absolute_error: 0.0171
Epoch 4/30
2839/2839 [==============================] - 145s 51ms/step - loss: 7.6809e-04 - mean_absolute_error: 0.0160
Epoch 5/30
2839/2839 [==============================] - 142s 50ms/step - loss: 3.6902e-04 - mean_absolute_error: 0.0129
Epoch 6/30
2839/2839 [==============================] - 142s 50ms/step - loss: 3.9476e-04 - mean_absolute_error: 0.0123
Epoch 7/30
2839/2839 [==============================] - 142s 50ms/step - loss: 3.3356e-04 - mean_absolute_error: 0.0114
Epoch 8/30
2839/2839 [==============================] - 142s 50ms/step - loss: 2.9207e-04 - mean_absolute_error: 0.0108
Epoch 9/30
2839/2839 [==============================] - 142s 50ms/step - loss: 2.7119e-04 - mean_absolute_error: 0.0099
Epoch 10/30
2839/2839 [==============================] - 142s 50ms/step - loss: 2.7803e-04 - mean_absolute_error: 0.0104
Epoch 11/30
2839/2839 [==============================] - 142s 50ms/step - loss: 4.0531e-04 - mean_absolute_error: 0.0087
Epoch 12/30
2839/2839 [==============================] - 142s 50ms/step - loss: 2.0770e-04 - mean_absolute_error: 0.0088
Epoch 13/30
2839/2839 [==============================] - 142s 50ms/step - loss: 1.6925e-04 - mean_absolute_error: 0.0081
Epoch 14/30
2839/2839 [==============================] - 142s 50ms/step - loss: 1.9385e-04 - mean_absolute_error: 0.0085
Epoch 15/30
2839/2839 [==============================] - 142s 50ms/step - loss: 1.7360e-04 - mean_absolute_error: 0.0088
Epoch 16/30
2839/2839 [==============================] - 143s 50ms/step - loss: 2.1322e-04 - mean_absolute_error: 0.0086
Epoch 17/30
2839/2839 [==============================] - 144s 51ms/step - loss: 1.6653e-04 - mean_absolute_error: 0.0083
Epoch 18/30
2839/2839 [==============================] - 145s 51ms/step - loss: 1.6397e-04 - mean_absolute_error: 0.0080
Epoch 19/30
2839/2839 [==============================] - 146s 51ms/step - loss: 1.7341e-04 - mean_absolute_error: 0.0085
Epoch 20/30
2839/2839 [==============================] - 145s 51ms/step - loss: 1.4799e-04 - mean_absolute_error: 0.0077
Epoch 21/30
2839/2839 [==============================] - 145s 51ms/step - loss: 1.4196e-04 - mean_absolute_error: 0.0078
Epoch 22/30
2839/2839 [==============================] - 145s 51ms/step - loss: 1.3255e-04 - mean_absolute_error: 0.0077
Epoch 23/30
2839/2839 [==============================] - 145s 51ms/step - loss: 1.6702e-04 - mean_absolute_error: 0.0080
Epoch 24/30
2839/2839 [==============================] - 145s 51ms/step - loss: 1.6953e-04 - mean_absolute_error: 0.0077
Epoch 25/30
2839/2839 [==============================] - 145s 51ms/step - loss: 1.3750e-04 - mean_absolute_error: 0.0076
Epoch 26/30
2839/2839 [==============================] - 145s 51ms/step - loss: 1.3372e-04 - mean_absolute_error: 0.0073
Epoch 27/30
2839/2839 [==============================] - 145s 51ms/step - loss: 1.5450e-04 - mean_absolute_error: 0.0080
Epoch 28/30
2839/2839 [==============================] - 145s 51ms/step - loss: 1.4270e-04 - mean_absolute_error: 0.0078
Epoch 29/30
2839/2839 [==============================] - 147s 52ms/step - loss: 1.4690e-04 - mean_absolute_error: 0.0076
Epoch 30/30
2839/2839 [==============================] - 142s 50ms/step - loss: 1.2227e-04 - mean_absolute_error: 0.0071

# Let's plot the mean_squared_error and mean_absolute_error using model history on the entire dataset
plot_loss_and_mae(
    model_name="model_8_16_16_32_64_8_8_1_30_6_months_overfit",
    model_history=history_8_16_16_32_64_8_8_1_30_6_months_overfit,
)

Observations

Using six months of data as a window to predict the stock price, training loss and mean absolute error are decreasing gradually. Increasing the number of LSTM layers has not resulted in a decrease in mean absolute error when compared to one LSTM and one dense layer, which was trained for 30 epochs. To potentially further reduce the mean absolute error, increasing/decreasing the number of LSTM layers or neurons may result in a decrease in MAE.

• 5 LSTM(8-16-16-32-64) 3 Dense(8-8-1) - 6 months
• Epochs: 30
• Batch Size: 1
• Training Loss: 1.2227e-04
• Mean Absolute Error: 0.0071

Model Building with 6 LSTM and 2 Dense layers(16-32-32-32-64-64-8-1) and 30 epochs - 3 months

# Let's create a keras sequential model
model_16_32_32_32_64_64_8_1_30_3_months_overfit = Sequential()

# Let's add two LSTM layer's to the model network
model_16_32_32_32_64_64_8_1_30_3_months_overfit.add(
    LSTM(16, return_sequences=True, input_shape=(x_train_3_months.shape[1], 1))
)
model_16_32_32_32_64_64_8_1_30_3_months_overfit.add(LSTM(32, return_sequences=True))
model_16_32_32_32_64_64_8_1_30_3_months_overfit.add(LSTM(32, return_sequences=True))
model_16_32_32_32_64_64_8_1_30_3_months_overfit.add(LSTM(32, return_sequences=True))
model_16_32_32_32_64_64_8_1_30_3_months_overfit.add(LSTM(64, return_sequences=True))
model_16_32_32_32_64_64_8_1_30_3_months_overfit.add(LSTM(64, return_sequences=False))


# Let's add dense layer to the model network
model_16_32_32_32_64_64_8_1_30_3_months_overfit.add(Dense(8))
model_16_32_32_32_64_64_8_1_30_3_months_overfit.add(Dense(1))

# Let's build the model using mean_squared_error as the loss function
# and mean_absolute_error as the evaluation metric during the compilation process
model_16_32_32_32_64_64_8_1_30_3_months_overfit.compile(
    optimizer="adam", loss="mean_squared_error", metrics=["mean_absolute_error"]
)

# Let's fit the sequential model with input features and output label
history_16_32_32_32_64_64_8_1_30_3_months_overfit = (
    model_16_32_32_32_64_64_8_1_30_3_months_overfit.fit(
        overall_x_train_3_months,
        overall_y_train_3_months,
        batch_size=1,
        epochs=30,
    )
)

Epoch 1/30
2929/2929 [==============================] - 98s 32ms/step - loss: 0.0028 - mean_absolute_error: 0.0290
Epoch 2/30
2929/2929 [==============================] - 95s 33ms/step - loss: 0.0011 - mean_absolute_error: 0.0212
Epoch 3/30
2929/2929 [==============================] - 96s 33ms/step - loss: 0.0014 - mean_absolute_error: 0.0219
Epoch 4/30
2929/2929 [==============================] - 96s 33ms/step - loss: 6.6454e-04 - mean_absolute_error: 0.0171
Epoch 5/30
2929/2929 [==============================] - 95s 33ms/step - loss: 5.3178e-04 - mean_absolute_error: 0.0147
Epoch 6/30
2929/2929 [==============================] - 95s 33ms/step - loss: 0.0045 - mean_absolute_error: 0.0196
Epoch 7/30
2929/2929 [==============================] - 95s 33ms/step - loss: 4.0227e-04 - mean_absolute_error: 0.0133
Epoch 8/30
2929/2929 [==============================] - 95s 33ms/step - loss: 3.7761e-04 - mean_absolute_error: 0.0137
Epoch 9/30
2929/2929 [==============================] - 96s 33ms/step - loss: 4.0484e-04 - mean_absolute_error: 0.0135
Epoch 10/30
2929/2929 [==============================] - 93s 32ms/step - loss: 4.6713e-04 - mean_absolute_error: 0.0134
Epoch 11/30
2929/2929 [==============================] - 93s 32ms/step - loss: 2.7554e-04 - mean_absolute_error: 0.0109
Epoch 12/30
2929/2929 [==============================] - 93s 32ms/step - loss: 3.3746e-04 - mean_absolute_error: 0.0115
Epoch 13/30
2929/2929 [==============================] - 93s 32ms/step - loss: 2.8992e-04 - mean_absolute_error: 0.0103
Epoch 14/30
2929/2929 [==============================] - 93s 32ms/step - loss: 2.3679e-04 - mean_absolute_error: 0.0095
Epoch 15/30
2929/2929 [==============================] - 93s 32ms/step - loss: 2.5169e-04 - mean_absolute_error: 0.0098
Epoch 16/30
2929/2929 [==============================] - 93s 32ms/step - loss: 2.2976e-04 - mean_absolute_error: 0.0091
Epoch 17/30
2929/2929 [==============================] - 94s 32ms/step - loss: 1.9134e-04 - mean_absolute_error: 0.0086
Epoch 18/30
2929/2929 [==============================] - 94s 32ms/step - loss: 2.2527e-04 - mean_absolute_error: 0.0088
Epoch 19/30
2929/2929 [==============================] - 92s 31ms/step - loss: 2.1156e-04 - mean_absolute_error: 0.0076
Epoch 20/30
2929/2929 [==============================] - 92s 31ms/step - loss: 1.6753e-04 - mean_absolute_error: 0.0081
Epoch 21/30
2929/2929 [==============================] - 92s 31ms/step - loss: 1.8042e-04 - mean_absolute_error: 0.0080
Epoch 22/30
2929/2929 [==============================] - 92s 31ms/step - loss: 2.1893e-04 - mean_absolute_error: 0.0090
Epoch 23/30
2929/2929 [==============================] - 92s 31ms/step - loss: 1.4054e-04 - mean_absolute_error: 0.0076
Epoch 24/30
2929/2929 [==============================] - 92s 31ms/step - loss: 1.6207e-04 - mean_absolute_error: 0.0083
Epoch 25/30
2929/2929 [==============================] - 92s 31ms/step - loss: 1.8724e-04 - mean_absolute_error: 0.0080
Epoch 26/30
2929/2929 [==============================] - 92s 31ms/step - loss: 2.0346e-04 - mean_absolute_error: 0.0083
Epoch 27/30
2929/2929 [==============================] - 92s 31ms/step - loss: 1.3592e-04 - mean_absolute_error: 0.0073
Epoch 28/30
2929/2929 [==============================] - 92s 31ms/step - loss: 1.8235e-04 - mean_absolute_error: 0.0078
Epoch 29/30
2929/2929 [==============================] - 92s 31ms/step - loss: 1.5957e-04 - mean_absolute_error: 0.0077
Epoch 30/30
2929/2929 [==============================] - 94s 32ms/step - loss: 1.8751e-04 - mean_absolute_error: 0.0073

# Let's plot the mean_squared_error and mean_absolute_error using model history on the entire dataset
plot_loss_and_mae(
    model_name="model_16_32_32_32_64_64_8_1_30_3_months_overfit",
    model_history=history_16_32_32_32_64_64_8_1_30_3_months_overfit,
)

Observations

Using three months of data as a window to predict the stock price, increasing the number of LSTM layers from five to six for 30 epochs, while reducing the number of dense layers from three to two, has not resulted in a decrease in the mean absolute error. Further steps, including increasing/decreasing the number of LSTM layers or neurons during training, might further reduce the mean absolute error.

• 6 LSTM(16-32-32-32-64-64) and 2 Dense(8-1) - 3 months
• Epochs: 30
• Batch Size: 1
• Training Loss: 1.8751e-04
• Mean Absolute Error: 0.0073

Model Building with 6 LSTM and 2 Dense layers(16-32-32-32-64-64-8-1) and 30 epochs - 6 months

# Let's create a keras sequential model
model_16_32_32_32_64_64_8_1_30_6_months_overfit = Sequential()

# Let's add two LSTM layer's to the model network
model_16_32_32_32_64_64_8_1_30_6_months_overfit.add(
    LSTM(16, return_sequences=True, input_shape=(x_train_6_months.shape[1], 1))
)
model_16_32_32_32_64_64_8_1_30_6_months_overfit.add(LSTM(32, return_sequences=True))
model_16_32_32_32_64_64_8_1_30_6_months_overfit.add(LSTM(32, return_sequences=True))
model_16_32_32_32_64_64_8_1_30_6_months_overfit.add(LSTM(32, return_sequences=True))
model_16_32_32_32_64_64_8_1_30_6_months_overfit.add(LSTM(64, return_sequences=True))
model_16_32_32_32_64_64_8_1_30_6_months_overfit.add(LSTM(64, return_sequences=False))


# Let's add dense layer to the model network
model_16_32_32_32_64_64_8_1_30_6_months_overfit.add(Dense(8))
model_16_32_32_32_64_64_8_1_30_6_months_overfit.add(Dense(1))

# Let's build the model using mean_squared_error as the loss function
# and mean_absolute_error as the evaluation metric during the compilation process
model_16_32_32_32_64_64_8_1_30_6_months_overfit.compile(
    optimizer="adam", loss="mean_squared_error", metrics=["mean_absolute_error"]
)

# Let's fit the sequential model with input features and output label
history_16_32_32_32_64_64_8_1_30_6_months_overfit = (
    model_16_32_32_32_64_64_8_1_30_6_months_overfit.fit(
        overall_x_train_6_months,
        overall_y_train_6_months,
        batch_size=1,
        epochs=30,
    )
)

Epoch 1/30
2839/2839 [==============================] - 188s 65ms/step - loss: 0.0029 - mean_absolute_error: 0.0306
Epoch 2/30
2839/2839 [==============================] - 185s 65ms/step - loss: 0.0013 - mean_absolute_error: 0.0228
Epoch 3/30
2839/2839 [==============================] - 184s 65ms/step - loss: 8.0480e-04 - mean_absolute_error: 0.0180
Epoch 4/30
2839/2839 [==============================] - 183s 65ms/step - loss: 6.8614e-04 - mean_absolute_error: 0.0159
Epoch 5/30
2839/2839 [==============================] - 187s 66ms/step - loss: 4.8392e-04 - mean_absolute_error: 0.0139
Epoch 6/30
2839/2839 [==============================] - 184s 65ms/step - loss: 4.3624e-04 - mean_absolute_error: 0.0133
Epoch 7/30
2839/2839 [==============================] - 200s 70ms/step - loss: 2.7267e-04 - mean_absolute_error: 0.0105
Epoch 8/30
2839/2839 [==============================] - 195s 69ms/step - loss: 3.5529e-04 - mean_absolute_error: 0.0112
Epoch 9/30
2839/2839 [==============================] - 187s 66ms/step - loss: 4.1537e-04 - mean_absolute_error: 0.0103
Epoch 10/30
2839/2839 [==============================] - 185s 65ms/step - loss: 1.7154e-04 - mean_absolute_error: 0.0088
Epoch 11/30
2839/2839 [==============================] - 186s 66ms/step - loss: 2.3544e-04 - mean_absolute_error: 0.0096
Epoch 12/30
2839/2839 [==============================] - 186s 66ms/step - loss: 2.3361e-04 - mean_absolute_error: 0.0101
Epoch 13/30
2839/2839 [==============================] - 184s 65ms/step - loss: 2.3748e-04 - mean_absolute_error: 0.0093
Epoch 14/30
2839/2839 [==============================] - 188s 66ms/step - loss: 1.9862e-04 - mean_absolute_error: 0.0092
Epoch 15/30
2839/2839 [==============================] - 190s 67ms/step - loss: 2.2353e-04 - mean_absolute_error: 0.0091
Epoch 16/30
2839/2839 [==============================] - 185s 65ms/step - loss: 2.1640e-04 - mean_absolute_error: 0.0094
Epoch 17/30
2839/2839 [==============================] - 186s 65ms/step - loss: 1.9582e-04 - mean_absolute_error: 0.0091
Epoch 18/30
2839/2839 [==============================] - 186s 65ms/step - loss: 2.3477e-04 - mean_absolute_error: 0.0087
Epoch 19/30
2839/2839 [==============================] - 204s 72ms/step - loss: 1.8918e-04 - mean_absolute_error: 0.0084
Epoch 20/30
2839/2839 [==============================] - 191s 67ms/step - loss: 2.3091e-04 - mean_absolute_error: 0.0098
Epoch 21/30
2839/2839 [==============================] - 187s 66ms/step - loss: 1.7635e-04 - mean_absolute_error: 0.0083
Epoch 22/30
2839/2839 [==============================] - 260s 92ms/step - loss: 2.0510e-04 - mean_absolute_error: 0.0087
Epoch 23/30
2839/2839 [==============================] - 313s 110ms/step - loss: 1.5988e-04 - mean_absolute_error: 0.0079
Epoch 24/30
2839/2839 [==============================] - 306s 108ms/step - loss: 1.6929e-04 - mean_absolute_error: 0.0082
Epoch 25/30
2839/2839 [==============================] - 305s 107ms/step - loss: 1.6552e-04 - mean_absolute_error: 0.0080
Epoch 26/30
2839/2839 [==============================] - 312s 110ms/step - loss: 1.4293e-04 - mean_absolute_error: 0.0078
Epoch 27/30
2839/2839 [==============================] - 310s 109ms/step - loss: 1.5859e-04 - mean_absolute_error: 0.0079
Epoch 28/30
2839/2839 [==============================] - 330s 116ms/step - loss: 1.7191e-04 - mean_absolute_error: 0.0078
Epoch 29/30
2839/2839 [==============================] - 326s 115ms/step - loss: 1.3630e-04 - mean_absolute_error: 0.0072
Epoch 30/30
2839/2839 [==============================] - 307s 108ms/step - loss: 1.5718e-04 - mean_absolute_error: 0.0079

# Let's plot the mean_squared_error and mean_absolute_error using model history on the entire dataset
plot_loss_and_mae(
    model_name="model_16_32_32_32_64_64_8_1_30_6_months_overfit",
    model_history=history_16_32_32_32_64_64_8_1_30_6_months_overfit,
)

Observations

Using six months of data as a window to predict the stock price, increasing the number of LSTM layers from five to six for 30 epochs, while reducing the number of dense layers from three to two, has not resulted in a decrease in the mean absolute error. Further steps, include increasing the number of neurons during training, might further reduce the mean absolute error.

• 6 LSTM(16-32-32-32-64-64) and 2 Dense(8-1) - 6 months
• Epochs: 30
• Batch Size: 1
• Training Loss: 1.5718e-04
• Mean Absolute Error: 0.0079

Increasing number of neurons in LSTM layers

Model Building with 3LSTM and 2Dense layers(64-128-128-8-1) and 30 epochs --> 3 months

# Let's create a keras sequential model
model_16_128_128_8_1_30_3_months_overfit = Sequential()

# Let's add two LSTM layer's to the model network
model_16_128_128_8_1_30_3_months_overfit.add(
    LSTM(64, return_sequences=True, input_shape=(x_train_3_months.shape[1], 1))
)
model_16_128_128_8_1_30_3_months_overfit.add(LSTM(128, return_sequences=True))
model_16_128_128_8_1_30_3_months_overfit.add(LSTM(128, return_sequences=False))

# Let's add dense layer to the model network
model_16_128_128_8_1_30_3_months_overfit.add(Dense(8))
model_16_128_128_8_1_30_3_months_overfit.add(Dense(1))

# Let's build the model using mean_squared_error as the loss function
# and mean_absolute_error as the evaluation metric during the compilation process
model_16_128_128_8_1_30_3_months_overfit.compile(
    optimizer="adam", loss="mean_squared_error", metrics=["mean_absolute_error"]
)

# Let's fit the sequential model with input features and output label
history_16_128_128_8_1_30_3_months_overfit = (
    model_16_128_128_8_1_30_3_months_overfit.fit(
        overall_x_train_3_months,
        overall_y_train_3_months,
        batch_size=1,
        epochs=30,
    )
)

Epoch 1/30
2929/2929 [==============================] - 128s 42ms/step - loss: 0.0017 - mean_absolute_error: 0.0192
Epoch 2/30
2929/2929 [==============================] - 120s 41ms/step - loss: 7.5301e-04 - mean_absolute_error: 0.0144
Epoch 3/30
2929/2929 [==============================] - 122s 42ms/step - loss: 7.3310e-04 - mean_absolute_error: 0.0123
Epoch 4/30
2929/2929 [==============================] - 123s 42ms/step - loss: 2.7189e-04 - mean_absolute_error: 0.0100
Epoch 5/30
2929/2929 [==============================] - 125s 43ms/step - loss: 2.3684e-04 - mean_absolute_error: 0.0097
Epoch 6/30
2929/2929 [==============================] - 132s 45ms/step - loss: 2.1949e-04 - mean_absolute_error: 0.0092
Epoch 7/30
2929/2929 [==============================] - 129s 44ms/step - loss: 2.9558e-04 - mean_absolute_error: 0.0097
Epoch 8/30
2929/2929 [==============================] - 127s 43ms/step - loss: 1.8197e-04 - mean_absolute_error: 0.0080
Epoch 9/30
2929/2929 [==============================] - 123s 42ms/step - loss: 1.4910e-04 - mean_absolute_error: 0.0075
Epoch 10/30
2929/2929 [==============================] - 121s 41ms/step - loss: 2.0143e-04 - mean_absolute_error: 0.0086
Epoch 11/30
2929/2929 [==============================] - 121s 41ms/step - loss: 5.2766e-04 - mean_absolute_error: 0.0085
Epoch 12/30
2929/2929 [==============================] - 122s 42ms/step - loss: 1.1250e-04 - mean_absolute_error: 0.0068
Epoch 13/30
2929/2929 [==============================] - 122s 42ms/step - loss: 1.6555e-04 - mean_absolute_error: 0.0079
Epoch 14/30
2929/2929 [==============================] - 121s 41ms/step - loss: 1.1762e-04 - mean_absolute_error: 0.0069
Epoch 15/30
2929/2929 [==============================] - 121s 41ms/step - loss: 1.2741e-04 - mean_absolute_error: 0.0070
Epoch 16/30
2929/2929 [==============================] - 121s 41ms/step - loss: 1.3823e-04 - mean_absolute_error: 0.0069
Epoch 17/30
2929/2929 [==============================] - 123s 42ms/step - loss: 1.2276e-04 - mean_absolute_error: 0.0069
Epoch 18/30
2929/2929 [==============================] - 121s 41ms/step - loss: 1.3388e-04 - mean_absolute_error: 0.0070
Epoch 19/30
2929/2929 [==============================] - 122s 41ms/step - loss: 1.2790e-04 - mean_absolute_error: 0.0070
Epoch 20/30
2929/2929 [==============================] - 122s 42ms/step - loss: 1.1680e-04 - mean_absolute_error: 0.0067
Epoch 21/30
2929/2929 [==============================] - 123s 42ms/step - loss: 1.2269e-04 - mean_absolute_error: 0.0072
Epoch 22/30
2929/2929 [==============================] - 120s 41ms/step - loss: 1.2557e-04 - mean_absolute_error: 0.0068
Epoch 23/30
2929/2929 [==============================] - 117s 40ms/step - loss: 9.9758e-05 - mean_absolute_error: 0.0064
Epoch 24/30
2929/2929 [==============================] - 118s 40ms/step - loss: 1.1430e-04 - mean_absolute_error: 0.0067
Epoch 25/30
2929/2929 [==============================] - 118s 40ms/step - loss: 1.3076e-04 - mean_absolute_error: 0.0067
Epoch 26/30
2929/2929 [==============================] - 117s 40ms/step - loss: 1.1126e-04 - mean_absolute_error: 0.0065
Epoch 27/30
2929/2929 [==============================] - 118s 40ms/step - loss: 1.1593e-04 - mean_absolute_error: 0.0066
Epoch 28/30
2929/2929 [==============================] - 126s 43ms/step - loss: 9.4204e-05 - mean_absolute_error: 0.0060
Epoch 29/30
2929/2929 [==============================] - 125s 43ms/step - loss: 1.0364e-04 - mean_absolute_error: 0.0064
Epoch 30/30
2929/2929 [==============================] - 125s 43ms/step - loss: 1.1014e-04 - mean_absolute_error: 0.0064

# Let's plot the mean_squared_error and mean_absolute_error using model history on the entire dataset
plot_loss_and_mae(
    model_name="model_16_128_128_8_1_30_3_months_overfit",
    model_history=history_16_128_128_8_1_30_3_months_overfit,
)

Observations

Using three months of data as a window to predict the stock price, increasing the number of neurons in LSTM layers has resulted in a decrease in the mean absolute error. Further steps, include increasing the number of both LSTM layers and neurons during training, might further reduce the mean absolute error.

• 3 LSTM(64-128-128) and 2 Dense(8-1) - 3 months
• Epochs: 30
• Batch Size: 1
• Training Loss: 1.1014e-04
• Mean Absolute Error: 0.0064

Model Building with 3LSTM and 2Dense layers(64-128-128-8-1) and 30 epochs --> 6 months

# Let's create a keras sequential model
model_16_128_128_8_1_30_6_months_overfit = Sequential()

# Let's add two LSTM layer's to the model network
model_16_128_128_8_1_30_6_months_overfit.add(
    LSTM(64, return_sequences=True, input_shape=(x_train_6_months.shape[1], 1))
)
model_16_128_128_8_1_30_6_months_overfit.add(LSTM(128, return_sequences=True))
model_16_128_128_8_1_30_6_months_overfit.add(LSTM(128, return_sequences=False))

# Let's add dense layer to the model network
model_16_128_128_8_1_30_6_months_overfit.add(Dense(8))
model_16_128_128_8_1_30_6_months_overfit.add(Dense(1))

# Let's build the model using mean_squared_error as the loss function
# and mean_absolute_error as the evaluation metric during the compilation process
model_16_128_128_8_1_30_6_months_overfit.compile(
    optimizer="adam", loss="mean_squared_error", metrics=["mean_absolute_error"]
)

# Let's fit the sequential model with input features and output label
history_16_128_128_8_1_30_6_months_overfit = (
    model_16_128_128_8_1_30_6_months_overfit.fit(
        overall_x_train_6_months,
        overall_y_train_6_months,
        batch_size=1,
        epochs=30,
    )
)

Epoch 1/30
2839/2839 [==============================] - 236s 82ms/step - loss: 0.0021 - mean_absolute_error: 0.0212
Epoch 2/30
2839/2839 [==============================] - 236s 83ms/step - loss: 5.4197e-04 - mean_absolute_error: 0.0127
Epoch 3/30
2839/2839 [==============================] - 235s 83ms/step - loss: 0.0023 - mean_absolute_error: 0.0131
Epoch 4/30
2839/2839 [==============================] - 239s 84ms/step - loss: 2.6934e-04 - mean_absolute_error: 0.0083
Epoch 5/30
2839/2839 [==============================] - 236s 83ms/step - loss: 2.9150e-04 - mean_absolute_error: 0.0096
Epoch 6/30
2839/2839 [==============================] - 243s 86ms/step - loss: 3.0606e-04 - mean_absolute_error: 0.0102
Epoch 7/30
2839/2839 [==============================] - 241s 85ms/step - loss: 2.5924e-04 - mean_absolute_error: 0.0098
Epoch 8/30
2839/2839 [==============================] - 237s 84ms/step - loss: 2.8250e-04 - mean_absolute_error: 0.0095
Epoch 9/30
2839/2839 [==============================] - 237s 83ms/step - loss: 2.9301e-04 - mean_absolute_error: 0.0092
Epoch 10/30
2839/2839 [==============================] - 240s 85ms/step - loss: 1.7889e-04 - mean_absolute_error: 0.0079
Epoch 11/30
2839/2839 [==============================] - 236s 83ms/step - loss: 2.8906e-04 - mean_absolute_error: 0.0095
Epoch 12/30
2839/2839 [==============================] - 232s 82ms/step - loss: 1.1985e-04 - mean_absolute_error: 0.0064
Epoch 13/30
2839/2839 [==============================] - 257s 91ms/step - loss: 1.8341e-04 - mean_absolute_error: 0.0081
Epoch 14/30
2839/2839 [==============================] - 254s 89ms/step - loss: 1.3751e-04 - mean_absolute_error: 0.0071
Epoch 15/30
2839/2839 [==============================] - 240s 85ms/step - loss: 1.3016e-04 - mean_absolute_error: 0.0072
Epoch 16/30
2839/2839 [==============================] - 234s 83ms/step - loss: 1.3566e-04 - mean_absolute_error: 0.0072
Epoch 17/30
2839/2839 [==============================] - 239s 84ms/step - loss: 3.7344e-04 - mean_absolute_error: 0.0066
Epoch 18/30
2839/2839 [==============================] - 205s 72ms/step - loss: 1.0044e-04 - mean_absolute_error: 0.0061
Epoch 19/30
2839/2839 [==============================] - 150s 53ms/step - loss: 1.1106e-04 - mean_absolute_error: 0.0068
Epoch 20/30
2839/2839 [==============================] - 197s 69ms/step - loss: 1.1422e-04 - mean_absolute_error: 0.0066
Epoch 21/30
2839/2839 [==============================] - 186s 65ms/step - loss: 1.1875e-04 - mean_absolute_error: 0.0067
Epoch 22/30
2839/2839 [==============================] - 142s 50ms/step - loss: 1.3079e-04 - mean_absolute_error: 0.0070
Epoch 23/30
2839/2839 [==============================] - 144s 51ms/step - loss: 1.0821e-04 - mean_absolute_error: 0.0065
Epoch 24/30
2839/2839 [==============================] - 144s 51ms/step - loss: 1.2966e-04 - mean_absolute_error: 0.0069
Epoch 25/30
2839/2839 [==============================] - 155s 55ms/step - loss: 1.1650e-04 - mean_absolute_error: 0.0067
Epoch 26/30
2839/2839 [==============================] - 149s 53ms/step - loss: 1.1301e-04 - mean_absolute_error: 0.0064
Epoch 27/30
2839/2839 [==============================] - 146s 51ms/step - loss: 1.1460e-04 - mean_absolute_error: 0.0068
Epoch 28/30
2839/2839 [==============================] - 145s 51ms/step - loss: 1.1550e-04 - mean_absolute_error: 0.0068
Epoch 29/30
2839/2839 [==============================] - 146s 52ms/step - loss: 1.0685e-04 - mean_absolute_error: 0.0064
Epoch 30/30
2839/2839 [==============================] - 156s 55ms/step - loss: 1.0524e-04 - mean_absolute_error: 0.0064

# Let's plot the mean_squared_error and mean_absolute_error using model history on the entire dataset
plot_loss_and_mae(
    model_name="model_16_128_128_8_1_30_6_months_overfit",
    model_history=history_16_128_128_8_1_30_6_months_overfit,
)

Observations

Using six months of data as a window to predict the stock price, increasing the number of neurons in LSTM layers has resulted in a decrease in the mean absolute error. Further steps, include increasing the number of both LSTM layers and neurons during training, might further reduce the mean absolute error.

• 3 LSTM(64-128-128) and 2 Dense(8-1) - 6 months
• Epochs: 30
• Batch Size: 1
• Training Loss: 1.0524e-04
• Mean Absolute Error: 0.0064

Model Building with 4 LSTM and 4 Dense layers (128-64-64-32-8-8-8-1) and 30 epochs - 3 months

# Let's create a keras sequential model
model_128_64_64_32_8_8_8_1_30_3_months_overfit = Sequential()

# Let's add two LSTM layer's to the model network
model_128_64_64_32_8_8_8_1_30_3_months_overfit.add(
    LSTM(128, return_sequences=True, input_shape=(x_train_3_months.shape[1], 1))
)
model_128_64_64_32_8_8_8_1_30_3_months_overfit.add(LSTM(64, return_sequences=True))
model_128_64_64_32_8_8_8_1_30_3_months_overfit.add(LSTM(64, return_sequences=True))
model_128_64_64_32_8_8_8_1_30_3_months_overfit.add(LSTM(32, return_sequences=False))

# Let's add dense layer to the model network
model_128_64_64_32_8_8_8_1_30_3_months_overfit.add(Dense(8))
model_128_64_64_32_8_8_8_1_30_3_months_overfit.add(Dense(8))
model_128_64_64_32_8_8_8_1_30_3_months_overfit.add(Dense(8))
model_128_64_64_32_8_8_8_1_30_3_months_overfit.add(Dense(1))

# Let's build the model using mean_squared_error as the loss function
# and mean_absolute_error as the evaluation metric during the compilation process
model_128_64_64_32_8_8_8_1_30_3_months_overfit.compile(
    optimizer="adam", loss="mean_squared_error", metrics=["mean_absolute_error"]
)

# Let's fit the sequential model with input features and output label
history_128_64_64_32_8_8_8_1_30_3_months_overfit = (
    model_128_64_64_32_8_8_8_1_30_3_months_overfit.fit(
        overall_x_train_3_months,
        overall_y_train_3_months,
        batch_size=1,
        epochs=30,
    )
)

Epoch 1/30
2929/2929 [==============================] - 82s 27ms/step - loss: 0.0024 - mean_absolute_error: 0.0256
Epoch 2/30
2929/2929 [==============================] - 79s 27ms/step - loss: 9.3084e-04 - mean_absolute_error: 0.0192
Epoch 3/30
2929/2929 [==============================] - 81s 28ms/step - loss: 0.0186 - mean_absolute_error: 0.0740
Epoch 4/30
2929/2929 [==============================] - 82s 28ms/step - loss: 0.0295 - mean_absolute_error: 0.1170
Epoch 5/30
2929/2929 [==============================] - 79s 27ms/step - loss: 0.0294 - mean_absolute_error: 0.1174
Epoch 6/30
2929/2929 [==============================] - 78s 27ms/step - loss: 0.0294 - mean_absolute_error: 0.1177
Epoch 7/30
2929/2929 [==============================] - 78s 27ms/step - loss: 0.0293 - mean_absolute_error: 0.1177
Epoch 8/30
2929/2929 [==============================] - 82s 28ms/step - loss: 0.0292 - mean_absolute_error: 0.1170
Epoch 9/30
2929/2929 [==============================] - 78s 27ms/step - loss: 0.0292 - mean_absolute_error: 0.1173
Epoch 10/30
2929/2929 [==============================] - 80s 27ms/step - loss: 0.0292 - mean_absolute_error: 0.1176
Epoch 11/30
2929/2929 [==============================] - 78s 27ms/step - loss: 0.0292 - mean_absolute_error: 0.1171
Epoch 12/30
2929/2929 [==============================] - 79s 27ms/step - loss: 0.0292 - mean_absolute_error: 0.1172
Epoch 13/30
2929/2929 [==============================] - 79s 27ms/step - loss: 0.0292 - mean_absolute_error: 0.1170
Epoch 14/30
2929/2929 [==============================] - 93s 32ms/step - loss: 0.0292 - mean_absolute_error: 0.1173
Epoch 15/30
2929/2929 [==============================] - 105s 36ms/step - loss: 0.0292 - mean_absolute_error: 0.1174
Epoch 16/30
2929/2929 [==============================] - 109s 37ms/step - loss: 0.0291 - mean_absolute_error: 0.1170
Epoch 17/30
2929/2929 [==============================] - 108s 37ms/step - loss: 0.0291 - mean_absolute_error: 0.1171
Epoch 18/30
2929/2929 [==============================] - 87s 30ms/step - loss: 0.0291 - mean_absolute_error: 0.1174
Epoch 19/30
2929/2929 [==============================] - 80s 27ms/step - loss: 0.0291 - mean_absolute_error: 0.1169
Epoch 20/30
2929/2929 [==============================] - 79s 27ms/step - loss: 0.0291 - mean_absolute_error: 0.1170
Epoch 21/30
2929/2929 [==============================] - 83s 28ms/step - loss: 0.0291 - mean_absolute_error: 0.1175
Epoch 22/30
2929/2929 [==============================] - 77s 26ms/step - loss: 0.0291 - mean_absolute_error: 0.1169
Epoch 23/30
2929/2929 [==============================] - 78s 26ms/step - loss: 0.0291 - mean_absolute_error: 0.1175
Epoch 24/30
2929/2929 [==============================] - 78s 27ms/step - loss: 0.0291 - mean_absolute_error: 0.1174
Epoch 25/30
2929/2929 [==============================] - 77s 26ms/step - loss: 0.0290 - mean_absolute_error: 0.1171
Epoch 26/30
2929/2929 [==============================] - 75s 26ms/step - loss: 0.0290 - mean_absolute_error: 0.1170
Epoch 27/30
2929/2929 [==============================] - 76s 26ms/step - loss: 0.0291 - mean_absolute_error: 0.1175
Epoch 28/30
2929/2929 [==============================] - 76s 26ms/step - loss: 0.0290 - mean_absolute_error: 0.1175
Epoch 29/30
2929/2929 [==============================] - 76s 26ms/step - loss: 0.0290 - mean_absolute_error: 0.1166
Epoch 30/30
2929/2929 [==============================] - 76s 26ms/step - loss: 0.0291 - mean_absolute_error: 0.1175

# Let's plot the mean_squared_error and mean_absolute_error using model history on the entire dataset
plot_loss_and_mae(
    model_name="model_128_64_64_32_8_8_8_1_30_3_months_overfit",
    model_history=history_128_64_64_32_8_8_8_1_30_3_months_overfit,
)

Observations

Using three months of data as a window to predict the stock price, training loss and mean absolute error are increasing gradually. Increasing the number of neurons in LSTM layers has resulted in an increase in mean absolute error when compared to one LSTM and one dense layer, which was trained for 30 epochs.

• 4 LSTM(128-64-64-32) 4 Dense(8-8-8-1)
• Epochs: 30
• Batch Size: 1
• Training Loss: 0.0291
• Mean Absolute Error: 0.1175

Model Building with 4 LSTM and 4 Dense layers (128-64-64-32-8-8-8-1) and 30 epochs - 6 months

# Let's create a keras sequential model
model_128_64_64_32_8_8_8_1_30_6_months_overfit = Sequential()

# Let's add two LSTM layer's to the model network
model_128_64_64_32_8_8_8_1_30_6_months_overfit.add(
    LSTM(128, return_sequences=True, input_shape=(x_train_6_months.shape[1], 1))
)
model_128_64_64_32_8_8_8_1_30_6_months_overfit.add(LSTM(64, return_sequences=True))
model_128_64_64_32_8_8_8_1_30_6_months_overfit.add(LSTM(64, return_sequences=True))
model_128_64_64_32_8_8_8_1_30_6_months_overfit.add(LSTM(32, return_sequences=False))

# Let's add dense layer to the model network
model_128_64_64_32_8_8_8_1_30_6_months_overfit.add(Dense(8))
model_128_64_64_32_8_8_8_1_30_6_months_overfit.add(Dense(8))
model_128_64_64_32_8_8_8_1_30_6_months_overfit.add(Dense(8))
model_128_64_64_32_8_8_8_1_30_6_months_overfit.add(Dense(1))

# Let's build the model using mean_squared_error as the loss function
# and mean_absolute_error as the evaluation metric during the compilation process
model_128_64_64_32_8_8_8_1_30_6_months_overfit.compile(
    optimizer="adam", loss="mean_squared_error", metrics=["mean_absolute_error"]
)

# Let's fit the sequential model with input features and output label
history_128_64_64_32_8_8_8_1_30_6_months_overfit = (
    model_128_64_64_32_8_8_8_1_30_6_months_overfit.fit(
        overall_x_train_6_months,
        overall_y_train_6_months,
        batch_size=1,
        epochs=30,
    )
)

Epoch 1/30
2839/2839 [==============================] - 154s 54ms/step - loss: 0.0020 - mean_absolute_error: 0.0247
Epoch 2/30
2839/2839 [==============================] - 152s 54ms/step - loss: 0.0013 - mean_absolute_error: 0.0211
Epoch 3/30
2839/2839 [==============================] - 152s 54ms/step - loss: 8.4741e-04 - mean_absolute_error: 0.0165
Epoch 4/30
2839/2839 [==============================] - 152s 53ms/step - loss: 4.5321e-04 - mean_absolute_error: 0.0139
Epoch 5/30
2839/2839 [==============================] - 154s 54ms/step - loss: 5.3346e-04 - mean_absolute_error: 0.0134
Epoch 6/30
2839/2839 [==============================] - 152s 54ms/step - loss: 3.0682e-04 - mean_absolute_error: 0.0112
Epoch 7/30
2839/2839 [==============================] - 155s 55ms/step - loss: 2.9179e-04 - mean_absolute_error: 0.0106
Epoch 8/30
2839/2839 [==============================] - 154s 54ms/step - loss: 2.4326e-04 - mean_absolute_error: 0.0100
Epoch 9/30
2839/2839 [==============================] - 153s 54ms/step - loss: 4.5337e-04 - mean_absolute_error: 0.0104
Epoch 10/30
2839/2839 [==============================] - 154s 54ms/step - loss: 1.5908e-04 - mean_absolute_error: 0.0085
Epoch 11/30
2839/2839 [==============================] - 156s 55ms/step - loss: 2.1867e-04 - mean_absolute_error: 0.0093
Epoch 12/30
2839/2839 [==============================] - 155s 54ms/step - loss: 2.6041e-04 - mean_absolute_error: 0.0103
Epoch 13/30
2839/2839 [==============================] - 155s 54ms/step - loss: 2.5302e-04 - mean_absolute_error: 0.0097
Epoch 14/30
2839/2839 [==============================] - 156s 55ms/step - loss: 1.6853e-04 - mean_absolute_error: 0.0084
Epoch 15/30
2839/2839 [==============================] - 154s 54ms/step - loss: 1.7335e-04 - mean_absolute_error: 0.0083
Epoch 16/30
2839/2839 [==============================] - 152s 54ms/step - loss: 2.0592e-04 - mean_absolute_error: 0.0088
Epoch 17/30
2839/2839 [==============================] - 158s 56ms/step - loss: 1.8143e-04 - mean_absolute_error: 0.0087
Epoch 18/30
2839/2839 [==============================] - 155s 54ms/step - loss: 1.8743e-04 - mean_absolute_error: 0.0086
Epoch 19/30
2839/2839 [==============================] - 154s 54ms/step - loss: 1.6161e-04 - mean_absolute_error: 0.0084
Epoch 20/30
2839/2839 [==============================] - 157s 55ms/step - loss: 1.6010e-04 - mean_absolute_error: 0.0083
Epoch 21/30
2839/2839 [==============================] - 158s 56ms/step - loss: 1.8595e-04 - mean_absolute_error: 0.0087
Epoch 22/30
2839/2839 [==============================] - 160s 56ms/step - loss: 1.5021e-04 - mean_absolute_error: 0.0078
Epoch 23/30
2839/2839 [==============================] - 160s 56ms/step - loss: 1.4725e-04 - mean_absolute_error: 0.0078
Epoch 24/30
2839/2839 [==============================] - 161s 57ms/step - loss: 4.2145e-04 - mean_absolute_error: 0.0084
Epoch 25/30
2839/2839 [==============================] - 155s 54ms/step - loss: 1.3728e-04 - mean_absolute_error: 0.0075
Epoch 26/30
2839/2839 [==============================] - 153s 54ms/step - loss: 1.5557e-04 - mean_absolute_error: 0.0077
Epoch 27/30
2839/2839 [==============================] - 153s 54ms/step - loss: 1.5361e-04 - mean_absolute_error: 0.0079
Epoch 28/30
2839/2839 [==============================] - 153s 54ms/step - loss: 1.4290e-04 - mean_absolute_error: 0.0078
Epoch 29/30
2839/2839 [==============================] - 153s 54ms/step - loss: 1.5010e-04 - mean_absolute_error: 0.0076
Epoch 30/30
2839/2839 [==============================] - 158s 56ms/step - loss: 2.0935e-04 - mean_absolute_error: 0.0085

# Let's plot the mean_squared_error and mean_absolute_error using model history on the entire dataset
plot_loss_and_mae(
    model_name="model_128_64_64_32_8_8_8_1_30_6_months_overfit",
    model_history=history_128_64_64_32_8_8_8_1_30_6_months_overfit,
)

Observations

Using six months of data as a window to predict the stock price, training loss and mean absolute error are increasing gradually. Increasing the number of neurons in LSTM layers has resulted in an increase in mean absolute error when compared to one LSTM and one dense layer, which was trained for 30 epochs.

• 4 LSTM(128-64-64-32) 4 Dense(8-8-8-1) - 6 months
• Epochs: 30
• Batch Size: 1
• Training Loss: 2.0935e-04
• Mean Absolute Error: 0.0085

Increasing Validation Accuracy

# Let's import keras package and set the logging status to ERROR
import keras

tf.compat.v1.logging.set_verbosity(tf.compat.v1.logging.ERROR)

# Function to plot the training and validation loss and mean absolute error
def plot_loss_and_mae_validation(model_name, model_history):
    figure, (ax1, ax2) = plt.subplots(
        1, 2, figsize=(18, 6)
    )  # Define a figure with two subplots
    figure.suptitle(model_name)  # Set the title of the figure

    # Plot the training and validation loss on the first subplot
    ax1.plot(
        model_history.history["loss"], label="Training Loss", color="blue"
    )  # Plot the training loss history
    ax1.plot(
        model_history.history["val_loss"], label="Validation Loss", color="red"
    )  # Plot the validation loss history
    ax1.set_title("Training and Validation Loss")  # Title for the loss plot
    ax1.set_xlabel("Epoch")  # Label for the x-axis
    ax1.set_ylabel("Loss")  # Label for the y-axis
    ax1.legend(loc="upper right")  # Legend positioned in the upper right

    # Plot the training and validation mean absolute error on the second subplot
    ax2.plot(
        model_history.history["mean_absolute_error"],
        label="Training MAE",
        color="green",
    )  # Plot the training MAE history
    ax2.plot(
        model_history.history["val_mean_absolute_error"],
        label="Validation MAE",
        color="orange",
    )  # Plot the validation MAE history
    ax2.set_title("Training and Validation MAE")  # Title for the MAE plot
    ax2.set_xlabel("Epoch")  # Label for the x-axis
    ax2.set_ylabel("Mean Absolute Error")  # Label for the y-axis
    ax2.legend(loc="upper right")  # Legend positioned in the upper right

    plt.show()  # Show the plot

Model Building with one LSTM layer and one Dense Layer (8-1) and 30 epochs - 3 months

# Let's create a keras sequential model
model_8_1_30_3_months_validation = Sequential()

# Let's add LSTM layer to the model network
model_8_1_30_3_months_validation.add(
    LSTM(8, return_sequences=False, input_shape=(x_train_3_months.shape[1], 1))
)
# Let's add dense layer to the model network
model_8_1_30_3_months_validation.add(Dense(1))

# Let's build the model using mean_squared_error as the loss function
# and mean_absolute_error as the evaluation metric during the compilation process
model_8_1_30_3_months_validation.compile(
    optimizer="adam", loss="mean_squared_error", metrics=["mean_absolute_error"]
)

# Let's define the callbacks for Model saving and Early stopping
cb_check = keras.callbacks.ModelCheckpoint(
    filepath="model_8_1_30_3_months_validation_checkpoint_filepath",
    save_best_only=True,
    monitor="val_loss",
)

cb_early = keras.callbacks.EarlyStopping(monitor="val_loss", patience=30)

# Let's fit the sequential model with input features and output label
history_8_1_30_3_months_validation = model_8_1_30_3_months_validation.fit(
    x_train_3_months,
    y_train_3_months,
    batch_size=16,
    epochs=30,
    validation_data=(x_valid_3_months, y_valid_3_months),
    callbacks=[cb_check, cb_early],
)

Epoch 1/30
146/146 [==============================] - 4s 18ms/step - loss: 0.0032 - mean_absolute_error: 0.0389 - val_loss: 0.0016 - val_mean_absolute_error: 0.0382
Epoch 2/30
146/146 [==============================] - 6s 38ms/step - loss: 7.8524e-05 - mean_absolute_error: 0.0055 - val_loss: 2.2856e-04 - val_mean_absolute_error: 0.0121
Epoch 3/30
146/146 [==============================] - 1s 8ms/step - loss: 2.5850e-05 - mean_absolute_error: 0.0034 - val_loss: 2.5316e-04 - val_mean_absolute_error: 0.0126
Epoch 4/30
146/146 [==============================] - 2s 16ms/step - loss: 2.1940e-05 - mean_absolute_error: 0.0030 - val_loss: 2.2610e-04 - val_mean_absolute_error: 0.0120
Epoch 5/30
146/146 [==============================] - 2s 16ms/step - loss: 2.0836e-05 - mean_absolute_error: 0.0028 - val_loss: 2.0072e-04 - val_mean_absolute_error: 0.0115
Epoch 6/30
146/146 [==============================] - 1s 8ms/step - loss: 1.8935e-05 - mean_absolute_error: 0.0026 - val_loss: 2.0363e-04 - val_mean_absolute_error: 0.0115
Epoch 7/30
146/146 [==============================] - 1s 8ms/step - loss: 1.8103e-05 - mean_absolute_error: 0.0024 - val_loss: 2.1212e-04 - val_mean_absolute_error: 0.0116
Epoch 8/30
146/146 [==============================] - 2s 15ms/step - loss: 1.8249e-05 - mean_absolute_error: 0.0025 - val_loss: 1.7708e-04 - val_mean_absolute_error: 0.0109
Epoch 9/30
146/146 [==============================] - 2s 15ms/step - loss: 1.7233e-05 - mean_absolute_error: 0.0024 - val_loss: 1.7305e-04 - val_mean_absolute_error: 0.0108
Epoch 10/30
146/146 [==============================] - 1s 8ms/step - loss: 1.7234e-05 - mean_absolute_error: 0.0025 - val_loss: 1.7402e-04 - val_mean_absolute_error: 0.0107
Epoch 11/30
146/146 [==============================] - 1s 8ms/step - loss: 1.6117e-05 - mean_absolute_error: 0.0022 - val_loss: 1.7450e-04 - val_mean_absolute_error: 0.0107
Epoch 12/30
146/146 [==============================] - 2s 15ms/step - loss: 1.5986e-05 - mean_absolute_error: 0.0022 - val_loss: 1.5963e-04 - val_mean_absolute_error: 0.0103
Epoch 13/30
146/146 [==============================] - 1s 8ms/step - loss: 1.7423e-05 - mean_absolute_error: 0.0025 - val_loss: 1.6503e-04 - val_mean_absolute_error: 0.0103
Epoch 14/30
146/146 [==============================] - 2s 16ms/step - loss: 1.5527e-05 - mean_absolute_error: 0.0022 - val_loss: 1.3677e-04 - val_mean_absolute_error: 0.0095
Epoch 15/30
146/146 [==============================] - 1s 10ms/step - loss: 1.5084e-05 - mean_absolute_error: 0.0023 - val_loss: 1.9629e-04 - val_mean_absolute_error: 0.0110
Epoch 16/30
146/146 [==============================] - 2s 16ms/step - loss: 1.5537e-05 - mean_absolute_error: 0.0023 - val_loss: 1.3043e-04 - val_mean_absolute_error: 0.0093
Epoch 17/30
146/146 [==============================] - 2s 14ms/step - loss: 1.4475e-05 - mean_absolute_error: 0.0022 - val_loss: 1.2785e-04 - val_mean_absolute_error: 0.0092
Epoch 18/30
146/146 [==============================] - 4s 30ms/step - loss: 1.4405e-05 - mean_absolute_error: 0.0022 - val_loss: 1.2342e-04 - val_mean_absolute_error: 0.0090
Epoch 19/30
146/146 [==============================] - 1s 8ms/step - loss: 1.3981e-05 - mean_absolute_error: 0.0022 - val_loss: 1.3089e-04 - val_mean_absolute_error: 0.0092
Epoch 20/30
146/146 [==============================] - 1s 7ms/step - loss: 1.3489e-05 - mean_absolute_error: 0.0021 - val_loss: 1.4488e-04 - val_mean_absolute_error: 0.0095
Epoch 21/30
146/146 [==============================] - 2s 15ms/step - loss: 1.4078e-05 - mean_absolute_error: 0.0023 - val_loss: 1.2191e-04 - val_mean_absolute_error: 0.0088
Epoch 22/30
146/146 [==============================] - 2s 15ms/step - loss: 1.4136e-05 - mean_absolute_error: 0.0023 - val_loss: 1.2152e-04 - val_mean_absolute_error: 0.0088
Epoch 23/30
146/146 [==============================] - 2s 15ms/step - loss: 1.2348e-05 - mean_absolute_error: 0.0020 - val_loss: 1.1752e-04 - val_mean_absolute_error: 0.0086
Epoch 24/30
146/146 [==============================] - 2s 15ms/step - loss: 1.1953e-05 - mean_absolute_error: 0.0019 - val_loss: 1.0263e-04 - val_mean_absolute_error: 0.0081
Epoch 25/30
146/146 [==============================] - 2s 15ms/step - loss: 1.2245e-05 - mean_absolute_error: 0.0020 - val_loss: 9.7097e-05 - val_mean_absolute_error: 0.0080
Epoch 26/30
146/146 [==============================] - 2s 14ms/step - loss: 1.2150e-05 - mean_absolute_error: 0.0021 - val_loss: 9.3384e-05 - val_mean_absolute_error: 0.0079
Epoch 27/30
146/146 [==============================] - 2s 15ms/step - loss: 1.1092e-05 - mean_absolute_error: 0.0019 - val_loss: 9.1404e-05 - val_mean_absolute_error: 0.0078
Epoch 28/30
146/146 [==============================] - 1s 8ms/step - loss: 1.1094e-05 - mean_absolute_error: 0.0019 - val_loss: 9.4677e-05 - val_mean_absolute_error: 0.0078
Epoch 29/30
146/146 [==============================] - 1s 8ms/step - loss: 1.1649e-05 - mean_absolute_error: 0.0020 - val_loss: 1.0225e-04 - val_mean_absolute_error: 0.0080
Epoch 30/30
146/146 [==============================] - 2s 14ms/step - loss: 1.0982e-05 - mean_absolute_error: 0.0020 - val_loss: 8.7737e-05 - val_mean_absolute_error: 0.0076

# Let's plot the mean_squared_error and mean_absolute_error using model history on the entire dataset
plot_loss_and_mae_validation(
    model_name="model_8_1_30_3_months_validation",
    model_history=history_8_1_30_3_months_validation,
)

Observations

Using three months of data as a window to predict the stock price, the training loss and training mean absolute error are decreasing gradually, whereas the validation mean absolute error is fluctuating, i.e., increasing and decreasing. One option to potentially decrease the validation MAE further is to increase or decrease the model's complexity by adding more layers or neurons.

• 1 LSTM(8) 1 Dense(1) - 3 months
• Epochs: 30
• Batch Size: 16
• Training Loss: 1.0982e-05
• Validation Loss: 8.7737e-05
• Training MAE: 0.0020
• Validation MAE: 0.0076

Model Building with one LSTM layer and one Dense Layer (8-1) and 30 epochs - 6 months

# Let's create a keras sequential model
model_8_1_30_6_months_validation = Sequential()

# Let's add LSTM layer to the model network
model_8_1_30_6_months_validation.add(
    LSTM(8, return_sequences=False, input_shape=(x_train_6_months.shape[1], 1))
)
# Let's add dense layer to the model network
model_8_1_30_6_months_validation.add(Dense(1))

# Let's build the model using mean_squared_error as the loss function
# and mean_absolute_error as the evaluation metric during the compilation process
model_8_1_30_6_months_validation.compile(
    optimizer="adam", loss="mean_squared_error", metrics=["mean_absolute_error"]
)

# Let's define the callbacks for Model saving and Early stopping
cb_check = keras.callbacks.ModelCheckpoint(
    filepath="model_8_1_30_6_months_validation_checkpoint_filepath",
    save_best_only=True,
    monitor="val_loss",
)

cb_early = keras.callbacks.EarlyStopping(monitor="val_loss", patience=30)

# Let's fit the sequential model with input features and output label
history_8_1_30_6_months_validation = model_8_1_30_6_months_validation.fit(
    x_train_6_months,
    y_train_6_months,
    batch_size=16,
    epochs=30,
    validation_data=(x_valid_6_months, y_valid_6_months),
    callbacks=[cb_check, cb_early],
)

Epoch 1/30
140/140 [==============================] - 4s 23ms/step - loss: 3.9203e-05 - mean_absolute_error: 0.0037 - val_loss: 2.4682e-04 - val_mean_absolute_error: 0.0126
Epoch 2/30
140/140 [==============================] - 3s 22ms/step - loss: 2.1954e-05 - mean_absolute_error: 0.0027 - val_loss: 2.2222e-04 - val_mean_absolute_error: 0.0119
Epoch 3/30
140/140 [==============================] - 3s 21ms/step - loss: 2.0909e-05 - mean_absolute_error: 0.0027 - val_loss: 1.8407e-04 - val_mean_absolute_error: 0.0112
Epoch 4/30
140/140 [==============================] - 3s 21ms/step - loss: 1.9613e-05 - mean_absolute_error: 0.0026 - val_loss: 1.7649e-04 - val_mean_absolute_error: 0.0108
Epoch 5/30
140/140 [==============================] - 3s 22ms/step - loss: 1.8573e-05 - mean_absolute_error: 0.0026 - val_loss: 1.6033e-04 - val_mean_absolute_error: 0.0104
Epoch 6/30
140/140 [==============================] - 2s 15ms/step - loss: 1.7676e-05 - mean_absolute_error: 0.0025 - val_loss: 1.7206e-04 - val_mean_absolute_error: 0.0104
Epoch 7/30
140/140 [==============================] - 3s 21ms/step - loss: 1.7114e-05 - mean_absolute_error: 0.0025 - val_loss: 1.5321e-04 - val_mean_absolute_error: 0.0101
Epoch 8/30
140/140 [==============================] - 3s 22ms/step - loss: 1.8061e-05 - mean_absolute_error: 0.0026 - val_loss: 1.4607e-04 - val_mean_absolute_error: 0.0097
Epoch 9/30
140/140 [==============================] - 2s 16ms/step - loss: 1.5064e-05 - mean_absolute_error: 0.0022 - val_loss: 1.4873e-04 - val_mean_absolute_error: 0.0097
Epoch 10/30
140/140 [==============================] - 3s 22ms/step - loss: 1.5272e-05 - mean_absolute_error: 0.0023 - val_loss: 1.2150e-04 - val_mean_absolute_error: 0.0090
Epoch 11/30
140/140 [==============================] - 4s 32ms/step - loss: 1.4962e-05 - mean_absolute_error: 0.0023 - val_loss: 1.1653e-04 - val_mean_absolute_error: 0.0088
Epoch 12/30
140/140 [==============================] - 3s 24ms/step - loss: 1.4709e-05 - mean_absolute_error: 0.0023 - val_loss: 1.1076e-04 - val_mean_absolute_error: 0.0086
Epoch 13/30
140/140 [==============================] - 3s 22ms/step - loss: 1.4185e-05 - mean_absolute_error: 0.0022 - val_loss: 1.0733e-04 - val_mean_absolute_error: 0.0084
Epoch 14/30
140/140 [==============================] - 3s 21ms/step - loss: 1.3084e-05 - mean_absolute_error: 0.0021 - val_loss: 1.0535e-04 - val_mean_absolute_error: 0.0083
Epoch 15/30
140/140 [==============================] - 3s 22ms/step - loss: 1.3311e-05 - mean_absolute_error: 0.0021 - val_loss: 9.9967e-05 - val_mean_absolute_error: 0.0081
Epoch 16/30
140/140 [==============================] - 3s 22ms/step - loss: 1.3776e-05 - mean_absolute_error: 0.0023 - val_loss: 9.9824e-05 - val_mean_absolute_error: 0.0082
Epoch 17/30
140/140 [==============================] - 3s 22ms/step - loss: 1.2426e-05 - mean_absolute_error: 0.0020 - val_loss: 9.2103e-05 - val_mean_absolute_error: 0.0079
Epoch 18/30
140/140 [==============================] - 3s 22ms/step - loss: 1.1942e-05 - mean_absolute_error: 0.0020 - val_loss: 8.8004e-05 - val_mean_absolute_error: 0.0077
Epoch 19/30
140/140 [==============================] - 3s 23ms/step - loss: 1.2342e-05 - mean_absolute_error: 0.0021 - val_loss: 8.6424e-05 - val_mean_absolute_error: 0.0076
Epoch 20/30
140/140 [==============================] - 4s 31ms/step - loss: 1.1339e-05 - mean_absolute_error: 0.0019 - val_loss: 8.2445e-05 - val_mean_absolute_error: 0.0075
Epoch 21/30
140/140 [==============================] - 3s 21ms/step - loss: 1.1698e-05 - mean_absolute_error: 0.0020 - val_loss: 7.9328e-05 - val_mean_absolute_error: 0.0074
Epoch 22/30
140/140 [==============================] - 3s 22ms/step - loss: 1.1643e-05 - mean_absolute_error: 0.0020 - val_loss: 7.8072e-05 - val_mean_absolute_error: 0.0073
Epoch 23/30
140/140 [==============================] - 3s 22ms/step - loss: 1.0834e-05 - mean_absolute_error: 0.0020 - val_loss: 7.4607e-05 - val_mean_absolute_error: 0.0072
Epoch 24/30
140/140 [==============================] - 3s 21ms/step - loss: 1.0380e-05 - mean_absolute_error: 0.0018 - val_loss: 7.4178e-05 - val_mean_absolute_error: 0.0071
Epoch 25/30
140/140 [==============================] - 3s 21ms/step - loss: 1.0712e-05 - mean_absolute_error: 0.0020 - val_loss: 7.1990e-05 - val_mean_absolute_error: 0.0071
Epoch 26/30
140/140 [==============================] - 2s 14ms/step - loss: 1.1144e-05 - mean_absolute_error: 0.0020 - val_loss: 7.2577e-05 - val_mean_absolute_error: 0.0070
Epoch 27/30
140/140 [==============================] - 3s 22ms/step - loss: 1.0511e-05 - mean_absolute_error: 0.0019 - val_loss: 7.0639e-05 - val_mean_absolute_error: 0.0069
Epoch 28/30
140/140 [==============================] - 2s 15ms/step - loss: 1.0209e-05 - mean_absolute_error: 0.0019 - val_loss: 7.1160e-05 - val_mean_absolute_error: 0.0069
Epoch 29/30
140/140 [==============================] - 3s 23ms/step - loss: 1.0335e-05 - mean_absolute_error: 0.0019 - val_loss: 6.7695e-05 - val_mean_absolute_error: 0.0069
Epoch 30/30
140/140 [==============================] - 2s 14ms/step - loss: 1.1369e-05 - mean_absolute_error: 0.0021 - val_loss: 6.8301e-05 - val_mean_absolute_error: 0.0068

# Let's plot the mean_squared_error and mean_absolute_error using model history on the entire dataset
plot_loss_and_mae_validation(
    model_name="model_8_1_30_3_months_validation",
    model_history=history_8_1_30_3_months_validation,
)

Observations

Using six months of data as a window to predict the stock price, the training loss and training mean absolute error are decreasing gradually, whereas the validation mean absolute error is fluctuating, i.e., increasing and decreasing. The observed validation MAE is lower compared to that using three months of data as a window. One option to potentially decrease the validation MAE further is to increase or decrease the model's complexity by adding more layers or neurons.

• 1 LSTM(8) 1 Dense(1) - 6 months
• Epochs: 30
• Batch Size: 16
• Training Loss: 1.1369e-05
• Validation Loss: 6.8301e-05
• Training MAE: 0.0021
• Validation MAE: 0.0068

Model Building with one LSTM layer and one Dense Layer (64-1) and 30 epochs - 3 months

# Let's create a keras sequential model
model_64_1_30_3_months_validation = Sequential()

# Let's add LSTM layer to the model network
model_64_1_30_3_months_validation.add(
    LSTM(64, return_sequences=False, input_shape=(x_train_3_months.shape[1], 1))
)
# Let's add dense layer to the model network
model_64_1_30_3_months_validation.add(Dense(1))

# Let's build the model using mean_squared_error as the loss function
# and mean_absolute_error as the evaluation metric during the compilation process
model_64_1_30_3_months_validation.compile(
    optimizer="adam", loss="mean_squared_error", metrics=["mean_absolute_error"]
)

# Let's define the callbacks for Model saving and Early stopping
cb_check = keras.callbacks.ModelCheckpoint(
    filepath="model_64_1_30_3_months_validation_checkpoint_filepath",
    save_best_only=True,
    monitor="val_loss",
)

cb_early = keras.callbacks.EarlyStopping(monitor="val_loss", patience=30)

# Let's fit the sequential model with input features and output label
history_64_1_30_3_months_validation = model_64_1_30_3_months_validation.fit(
    x_train_3_months,
    y_train_3_months,
    batch_size=16,
    epochs=30,
    callbacks=[cb_check, cb_early],
    validation_data=(x_valid_3_months, y_valid_3_months),
)

Epoch 1/30
146/146 [==============================] - 5s 30ms/step - loss: 2.1252e-04 - mean_absolute_error: 0.0066 - val_loss: 2.2478e-04 - val_mean_absolute_error: 0.0124
Epoch 2/30
146/146 [==============================] - 3s 19ms/step - loss: 2.7293e-05 - mean_absolute_error: 0.0033 - val_loss: 2.1015e-04 - val_mean_absolute_error: 0.0118
Epoch 3/30
146/146 [==============================] - 2s 13ms/step - loss: 2.3973e-05 - mean_absolute_error: 0.0031 - val_loss: 2.2477e-04 - val_mean_absolute_error: 0.0118
Epoch 4/30
146/146 [==============================] - 3s 18ms/step - loss: 2.1215e-05 - mean_absolute_error: 0.0028 - val_loss: 1.4018e-04 - val_mean_absolute_error: 0.0096
Epoch 5/30
146/146 [==============================] - 3s 18ms/step - loss: 1.8734e-05 - mean_absolute_error: 0.0026 - val_loss: 1.3327e-04 - val_mean_absolute_error: 0.0094
Epoch 6/30
146/146 [==============================] - 2s 12ms/step - loss: 1.6463e-05 - mean_absolute_error: 0.0025 - val_loss: 2.5561e-04 - val_mean_absolute_error: 0.0128
Epoch 7/30
146/146 [==============================] - 3s 18ms/step - loss: 1.9914e-05 - mean_absolute_error: 0.0031 - val_loss: 1.0884e-04 - val_mean_absolute_error: 0.0085
Epoch 8/30
146/146 [==============================] - 2s 12ms/step - loss: 1.4131e-05 - mean_absolute_error: 0.0022 - val_loss: 1.1052e-04 - val_mean_absolute_error: 0.0084
Epoch 9/30
146/146 [==============================] - 3s 18ms/step - loss: 1.4899e-05 - mean_absolute_error: 0.0024 - val_loss: 1.0201e-04 - val_mean_absolute_error: 0.0082
Epoch 10/30
146/146 [==============================] - 2s 12ms/step - loss: 1.8597e-05 - mean_absolute_error: 0.0029 - val_loss: 1.5155e-04 - val_mean_absolute_error: 0.0096
Epoch 11/30
146/146 [==============================] - 3s 18ms/step - loss: 1.4302e-05 - mean_absolute_error: 0.0023 - val_loss: 9.4915e-05 - val_mean_absolute_error: 0.0081
Epoch 12/30
146/146 [==============================] - 2s 12ms/step - loss: 1.3310e-05 - mean_absolute_error: 0.0023 - val_loss: 1.0690e-04 - val_mean_absolute_error: 0.0082
Epoch 13/30
146/146 [==============================] - 2s 13ms/step - loss: 1.1826e-05 - mean_absolute_error: 0.0021 - val_loss: 1.0256e-04 - val_mean_absolute_error: 0.0080
Epoch 14/30
146/146 [==============================] - 3s 19ms/step - loss: 1.1504e-05 - mean_absolute_error: 0.0021 - val_loss: 8.8195e-05 - val_mean_absolute_error: 0.0078
Epoch 15/30
146/146 [==============================] - 3s 19ms/step - loss: 1.1417e-05 - mean_absolute_error: 0.0021 - val_loss: 8.3068e-05 - val_mean_absolute_error: 0.0076
Epoch 16/30
146/146 [==============================] - 2s 12ms/step - loss: 1.2056e-05 - mean_absolute_error: 0.0021 - val_loss: 1.0493e-04 - val_mean_absolute_error: 0.0080
Epoch 17/30
146/146 [==============================] - 2s 12ms/step - loss: 1.2309e-05 - mean_absolute_error: 0.0022 - val_loss: 8.6657e-05 - val_mean_absolute_error: 0.0076
Epoch 18/30
146/146 [==============================] - 3s 19ms/step - loss: 9.7148e-06 - mean_absolute_error: 0.0018 - val_loss: 7.9313e-05 - val_mean_absolute_error: 0.0075
Epoch 19/30
146/146 [==============================] - 2s 13ms/step - loss: 1.1721e-05 - mean_absolute_error: 0.0022 - val_loss: 8.0040e-05 - val_mean_absolute_error: 0.0074
Epoch 20/30
146/146 [==============================] - 4s 27ms/step - loss: 1.2142e-05 - mean_absolute_error: 0.0023 - val_loss: 7.5503e-05 - val_mean_absolute_error: 0.0072
Epoch 21/30
146/146 [==============================] - 2s 12ms/step - loss: 9.0522e-06 - mean_absolute_error: 0.0018 - val_loss: 7.7642e-05 - val_mean_absolute_error: 0.0075
Epoch 22/30
146/146 [==============================] - 2s 11ms/step - loss: 9.4761e-06 - mean_absolute_error: 0.0019 - val_loss: 7.7329e-05 - val_mean_absolute_error: 0.0072
Epoch 23/30
146/146 [==============================] - 2s 12ms/step - loss: 1.0467e-05 - mean_absolute_error: 0.0021 - val_loss: 8.0157e-05 - val_mean_absolute_error: 0.0073
Epoch 24/30
146/146 [==============================] - 3s 19ms/step - loss: 1.2376e-05 - mean_absolute_error: 0.0023 - val_loss: 7.1307e-05 - val_mean_absolute_error: 0.0071
Epoch 25/30
146/146 [==============================] - 3s 19ms/step - loss: 9.9082e-06 - mean_absolute_error: 0.0020 - val_loss: 7.0420e-05 - val_mean_absolute_error: 0.0071
Epoch 26/30
146/146 [==============================] - 3s 18ms/step - loss: 8.9151e-06 - mean_absolute_error: 0.0018 - val_loss: 6.7120e-05 - val_mean_absolute_error: 0.0069
Epoch 27/30
146/146 [==============================] - 2s 12ms/step - loss: 9.7693e-06 - mean_absolute_error: 0.0020 - val_loss: 6.7955e-05 - val_mean_absolute_error: 0.0069
Epoch 28/30
146/146 [==============================] - 2s 11ms/step - loss: 8.7222e-06 - mean_absolute_error: 0.0018 - val_loss: 6.8253e-05 - val_mean_absolute_error: 0.0068
Epoch 29/30
146/146 [==============================] - 2s 11ms/step - loss: 9.9959e-06 - mean_absolute_error: 0.0019 - val_loss: 6.9555e-05 - val_mean_absolute_error: 0.0070
Epoch 30/30
146/146 [==============================] - 3s 18ms/step - loss: 9.7072e-06 - mean_absolute_error: 0.0019 - val_loss: 6.5499e-05 - val_mean_absolute_error: 0.0067

# Let's plot the mean_squared_error and mean_absolute_error using model history on the entire dataset
plot_loss_and_mae_validation(
    model_name="model_64_1_30_3_months_validation",
    model_history=history_64_1_30_3_months_validation,
)

Observations

Using three months of data as a window to predict the stock price, the training loss and training mean absolute error are decreasing gradually, whereas the validation loss and validation mean absolute error are fluctuating, i.e., increasing and decreasing. The observed validation MAE is lower compared to that using one LSTM layer with one neuron. One option to potentially decrease the validation MAE further is to increase or decrease the model's complexity by adding/reducing more layers or neurons.

• 1 LSTM(64) 1 Dense(1) - 3 months
• Epochs: 30
• Batch Size: 16
• Training Loss: 9.7072e-06
• Validation Loss: 6.5499e-05
• Training MAE: 0.0019
• Validation MAE: 0.0067

Model Building with one LSTM layer and one Dense Layer (64-1) and 30 epochs - 6 months

# Let's create a keras sequential model
model_64_1_30_6_months_validation = Sequential()

# Let's add LSTM layer to the model network
model_64_1_30_6_months_validation.add(
    LSTM(64, return_sequences=False, input_shape=(x_train_6_months.shape[1], 1))
)
# Let's add dense layer to the model network
model_64_1_30_6_months_validation.add(Dense(1))

# Let's build the model using mean_squared_error as the loss function
# and mean_absolute_error as the evaluation metric during the compilation process
model_64_1_30_6_months_validation.compile(
    optimizer="adam", loss="mean_squared_error", metrics=["mean_absolute_error"]
)

# Let's define the callbacks for Model saving and Early stopping
cb_check = keras.callbacks.ModelCheckpoint(
    filepath="model_64_1_30_6_months_validation_checkpoint_filepath",
    save_best_only=True,
    monitor="val_loss",
)

cb_early = keras.callbacks.EarlyStopping(monitor="val_loss", patience=30)

# Let's fit the sequential model with input features and output label
history_64_1_30_6_months_validation = model_64_1_30_6_months_validation.fit(
    x_train_6_months,
    y_train_6_months,
    batch_size=16,
    epochs=30,
    callbacks=[cb_check, cb_early],
    validation_data=(x_valid_6_months, y_valid_6_months),
)

Epoch 1/30
140/140 [==============================] - 5s 32ms/step - loss: 1.8658e-04 - mean_absolute_error: 0.0065 - val_loss: 2.3132e-04 - val_mean_absolute_error: 0.0125
Epoch 2/30
140/140 [==============================] - 4s 30ms/step - loss: 2.8214e-05 - mean_absolute_error: 0.0032 - val_loss: 1.9370e-04 - val_mean_absolute_error: 0.0114
Epoch 3/30
140/140 [==============================] - 4s 30ms/step - loss: 2.4355e-05 - mean_absolute_error: 0.0031 - val_loss: 1.7152e-04 - val_mean_absolute_error: 0.0105
Epoch 4/30
140/140 [==============================] - 5s 37ms/step - loss: 2.8602e-05 - mean_absolute_error: 0.0035 - val_loss: 1.6264e-04 - val_mean_absolute_error: 0.0103
Epoch 5/30
140/140 [==============================] - 4s 29ms/step - loss: 1.8020e-05 - mean_absolute_error: 0.0026 - val_loss: 1.6054e-04 - val_mean_absolute_error: 0.0100
Epoch 6/30
140/140 [==============================] - 4s 29ms/step - loss: 1.9152e-05 - mean_absolute_error: 0.0027 - val_loss: 1.1625e-04 - val_mean_absolute_error: 0.0088
Epoch 7/30
140/140 [==============================] - 3s 24ms/step - loss: 1.7034e-05 - mean_absolute_error: 0.0026 - val_loss: 1.4075e-04 - val_mean_absolute_error: 0.0093
Epoch 8/30
140/140 [==============================] - 4s 30ms/step - loss: 1.5620e-05 - mean_absolute_error: 0.0024 - val_loss: 1.0837e-04 - val_mean_absolute_error: 0.0084
Epoch 9/30
140/140 [==============================] - 3s 22ms/step - loss: 1.6835e-05 - mean_absolute_error: 0.0025 - val_loss: 1.1131e-04 - val_mean_absolute_error: 0.0084
Epoch 10/30
140/140 [==============================] - 4s 30ms/step - loss: 1.4573e-05 - mean_absolute_error: 0.0023 - val_loss: 1.0149e-04 - val_mean_absolute_error: 0.0081
Epoch 11/30
140/140 [==============================] - 4s 29ms/step - loss: 1.6477e-05 - mean_absolute_error: 0.0026 - val_loss: 8.3379e-05 - val_mean_absolute_error: 0.0077
Epoch 12/30
140/140 [==============================] - 4s 31ms/step - loss: 1.3990e-05 - mean_absolute_error: 0.0023 - val_loss: 8.2960e-05 - val_mean_absolute_error: 0.0077
Epoch 13/30
140/140 [==============================] - 4s 30ms/step - loss: 1.3502e-05 - mean_absolute_error: 0.0023 - val_loss: 7.8199e-05 - val_mean_absolute_error: 0.0074
Epoch 14/30
140/140 [==============================] - 4s 30ms/step - loss: 1.2276e-05 - mean_absolute_error: 0.0021 - val_loss: 7.3984e-05 - val_mean_absolute_error: 0.0073
Epoch 15/30
140/140 [==============================] - 3s 24ms/step - loss: 1.3256e-05 - mean_absolute_error: 0.0022 - val_loss: 1.0024e-04 - val_mean_absolute_error: 0.0083
Epoch 16/30
140/140 [==============================] - 4s 29ms/step - loss: 1.2398e-05 - mean_absolute_error: 0.0022 - val_loss: 7.0872e-05 - val_mean_absolute_error: 0.0072
Epoch 17/30
140/140 [==============================] - 5s 38ms/step - loss: 1.3191e-05 - mean_absolute_error: 0.0023 - val_loss: 6.9975e-05 - val_mean_absolute_error: 0.0072
Epoch 18/30
140/140 [==============================] - 3s 22ms/step - loss: 1.1999e-05 - mean_absolute_error: 0.0022 - val_loss: 1.1898e-04 - val_mean_absolute_error: 0.0085
Epoch 19/30
140/140 [==============================] - 4s 29ms/step - loss: 1.4114e-05 - mean_absolute_error: 0.0024 - val_loss: 6.3551e-05 - val_mean_absolute_error: 0.0069
Epoch 20/30
140/140 [==============================] - 3s 22ms/step - loss: 1.1502e-05 - mean_absolute_error: 0.0021 - val_loss: 6.4559e-05 - val_mean_absolute_error: 0.0069
Epoch 21/30
140/140 [==============================] - 4s 30ms/step - loss: 1.2650e-05 - mean_absolute_error: 0.0022 - val_loss: 6.2989e-05 - val_mean_absolute_error: 0.0069
Epoch 22/30
140/140 [==============================] - 3s 22ms/step - loss: 1.0539e-05 - mean_absolute_error: 0.0020 - val_loss: 6.9902e-05 - val_mean_absolute_error: 0.0067
Epoch 23/30
140/140 [==============================] - 4s 28ms/step - loss: 1.0606e-05 - mean_absolute_error: 0.0020 - val_loss: 6.1798e-05 - val_mean_absolute_error: 0.0068
Epoch 24/30
140/140 [==============================] - 4s 32ms/step - loss: 8.8666e-06 - mean_absolute_error: 0.0018 - val_loss: 6.1589e-05 - val_mean_absolute_error: 0.0068
Epoch 25/30
140/140 [==============================] - 4s 30ms/step - loss: 1.1765e-05 - mean_absolute_error: 0.0022 - val_loss: 5.8969e-05 - val_mean_absolute_error: 0.0066
Epoch 26/30
140/140 [==============================] - 3s 23ms/step - loss: 8.7927e-06 - mean_absolute_error: 0.0018 - val_loss: 7.1840e-05 - val_mean_absolute_error: 0.0067
Epoch 27/30
140/140 [==============================] - 4s 31ms/step - loss: 1.0307e-05 - mean_absolute_error: 0.0020 - val_loss: 5.5305e-05 - val_mean_absolute_error: 0.0063
Epoch 28/30
140/140 [==============================] - 3s 23ms/step - loss: 8.8721e-06 - mean_absolute_error: 0.0019 - val_loss: 6.6018e-05 - val_mean_absolute_error: 0.0064
Epoch 29/30
140/140 [==============================] - 4s 31ms/step - loss: 1.0117e-05 - mean_absolute_error: 0.0021 - val_loss: 5.3099e-05 - val_mean_absolute_error: 0.0061
Epoch 30/30
140/140 [==============================] - 4s 30ms/step - loss: 8.5071e-06 - mean_absolute_error: 0.0017 - val_loss: 5.2503e-05 - val_mean_absolute_error: 0.0061

# Let's plot the mean_squared_error and mean_absolute_error using model history on the entire dataset
plot_loss_and_mae_validation(
    model_name="model_64_1_30_6_months_validation",
    model_history=history_64_1_30_6_months_validation,
)

Observations

Using six months of data as a window to predict the stock price, the training loss and training mean absolute error are decreasing gradually, whereas the validation loss and validation mean absolute error are fluctuating, i.e., increasing and decreasing. The observed validation MAE is lower compared to that using three months of data as a window. One option to potentially further decrease the validation MAE is to increase or decrease the model's complexity by adding or reducing layers or neurons.

• 1 LSTM(64) 1 Dense(1) - 6 months
• Epochs: 30
• Batch Size: 16
• Training Loss: 8.5071e-06
• Validation Loss: 5.2503e-05
• Training MAE: 0.0017
• Validation MAE: 0.0061

Model Building with one LSTM layer and Dense layer (128-1) and 30 epochs - 3 months

# Let's create a keras sequential model
model_128_1_30_3_months_validation = Sequential()

# Let's add LSTM layer to the model network
model_128_1_30_3_months_validation.add(
    LSTM(128, return_sequences=False, input_shape=(x_train_3_months.shape[1], 1))
)
# Let's add dense layer to the model network
model_128_1_30_3_months_validation.add(Dense(1))

# Let's build the model using mean_squared_error as the loss function
# and mean_absolute_error as the evaluation metric during the compilation process
model_128_1_30_3_months_validation.compile(
    optimizer="adam", loss="mean_squared_error", metrics=["mean_absolute_error"]
)

# Let's define the callbacks for Model saving and Early stopping
cb_check = keras.callbacks.ModelCheckpoint(
    filepath="model_128_1_30_3_months_validation_checkpoint_filepath",
    save_best_only=True,
    monitor="val_loss",
)

cb_early = keras.callbacks.EarlyStopping(monitor="val_loss", patience=30)

# Let's fit the sequential model with input features and output label
history_128_1_30_3_months_validation = model_128_1_30_3_months_validation.fit(
    x_train_3_months,
    y_train_3_months,
    batch_size=16,
    epochs=30,
    callbacks=[cb_check, cb_early],
    validation_data=(x_valid_3_months, y_valid_3_months),
)

Epoch 1/30
146/146 [==============================] - 7s 41ms/step - loss: 2.3608e-04 - mean_absolute_error: 0.0066 - val_loss: 2.1569e-04 - val_mean_absolute_error: 0.0119
Epoch 2/30
146/146 [==============================] - 4s 25ms/step - loss: 2.7367e-05 - mean_absolute_error: 0.0032 - val_loss: 2.1988e-04 - val_mean_absolute_error: 0.0117
Epoch 3/30
146/146 [==============================] - 5s 31ms/step - loss: 2.1176e-05 - mean_absolute_error: 0.0028 - val_loss: 1.4830e-04 - val_mean_absolute_error: 0.0098
Epoch 4/30
146/146 [==============================] - 4s 31ms/step - loss: 2.1537e-05 - mean_absolute_error: 0.0030 - val_loss: 1.3202e-04 - val_mean_absolute_error: 0.0092
Epoch 5/30
146/146 [==============================] - 5s 31ms/step - loss: 2.0688e-05 - mean_absolute_error: 0.0029 - val_loss: 1.2300e-04 - val_mean_absolute_error: 0.0090
Epoch 6/30
146/146 [==============================] - 4s 24ms/step - loss: 1.8999e-05 - mean_absolute_error: 0.0027 - val_loss: 1.6314e-04 - val_mean_absolute_error: 0.0099
Epoch 7/30
146/146 [==============================] - 4s 24ms/step - loss: 2.0325e-05 - mean_absolute_error: 0.0031 - val_loss: 1.5771e-04 - val_mean_absolute_error: 0.0105
Epoch 8/30
146/146 [==============================] - 3s 24ms/step - loss: 1.6727e-05 - mean_absolute_error: 0.0026 - val_loss: 1.2465e-04 - val_mean_absolute_error: 0.0088
Epoch 9/30
146/146 [==============================] - 3s 24ms/step - loss: 1.6664e-05 - mean_absolute_error: 0.0026 - val_loss: 1.3846e-04 - val_mean_absolute_error: 0.0099
Epoch 10/30
146/146 [==============================] - 5s 32ms/step - loss: 1.3694e-05 - mean_absolute_error: 0.0022 - val_loss: 1.0307e-04 - val_mean_absolute_error: 0.0085
Epoch 11/30
146/146 [==============================] - 4s 31ms/step - loss: 1.4434e-05 - mean_absolute_error: 0.0024 - val_loss: 9.7900e-05 - val_mean_absolute_error: 0.0082
Epoch 12/30
146/146 [==============================] - 4s 31ms/step - loss: 1.3772e-05 - mean_absolute_error: 0.0023 - val_loss: 9.5211e-05 - val_mean_absolute_error: 0.0081
Epoch 13/30
146/146 [==============================] - 4s 26ms/step - loss: 1.4357e-05 - mean_absolute_error: 0.0024 - val_loss: 1.9242e-04 - val_mean_absolute_error: 0.0108
Epoch 14/30
146/146 [==============================] - 4s 24ms/step - loss: 1.2602e-05 - mean_absolute_error: 0.0022 - val_loss: 9.7082e-05 - val_mean_absolute_error: 0.0080
Epoch 15/30
146/146 [==============================] - 4s 31ms/step - loss: 1.2225e-05 - mean_absolute_error: 0.0021 - val_loss: 9.1706e-05 - val_mean_absolute_error: 0.0079
Epoch 16/30
146/146 [==============================] - 4s 25ms/step - loss: 1.3564e-05 - mean_absolute_error: 0.0024 - val_loss: 9.3813e-05 - val_mean_absolute_error: 0.0081
Epoch 17/30
146/146 [==============================] - 4s 25ms/step - loss: 1.0203e-05 - mean_absolute_error: 0.0019 - val_loss: 1.4699e-04 - val_mean_absolute_error: 0.0100
Epoch 18/30
146/146 [==============================] - 4s 26ms/step - loss: 1.1531e-05 - mean_absolute_error: 0.0021 - val_loss: 9.8101e-05 - val_mean_absolute_error: 0.0083
Epoch 19/30
146/146 [==============================] - 4s 25ms/step - loss: 1.0288e-05 - mean_absolute_error: 0.0019 - val_loss: 9.6364e-05 - val_mean_absolute_error: 0.0078
Epoch 20/30
146/146 [==============================] - 4s 24ms/step - loss: 1.1878e-05 - mean_absolute_error: 0.0022 - val_loss: 9.2091e-05 - val_mean_absolute_error: 0.0076
Epoch 21/30
146/146 [==============================] - 5s 35ms/step - loss: 1.0568e-05 - mean_absolute_error: 0.0020 - val_loss: 7.5058e-05 - val_mean_absolute_error: 0.0073
Epoch 22/30
146/146 [==============================] - 4s 28ms/step - loss: 1.2142e-05 - mean_absolute_error: 0.0022 - val_loss: 8.4710e-05 - val_mean_absolute_error: 0.0077
Epoch 23/30
146/146 [==============================] - 4s 26ms/step - loss: 9.7793e-06 - mean_absolute_error: 0.0020 - val_loss: 9.1394e-05 - val_mean_absolute_error: 0.0075
Epoch 24/30
146/146 [==============================] - 7s 46ms/step - loss: 1.1459e-05 - mean_absolute_error: 0.0022 - val_loss: 7.3919e-05 - val_mean_absolute_error: 0.0070
Epoch 25/30
146/146 [==============================] - 4s 26ms/step - loss: 9.0986e-06 - mean_absolute_error: 0.0017 - val_loss: 7.6761e-05 - val_mean_absolute_error: 0.0071
Epoch 26/30
146/146 [==============================] - 5s 33ms/step - loss: 1.2795e-05 - mean_absolute_error: 0.0024 - val_loss: 6.6168e-05 - val_mean_absolute_error: 0.0069
Epoch 27/30
146/146 [==============================] - 5s 36ms/step - loss: 1.0708e-05 - mean_absolute_error: 0.0022 - val_loss: 6.3159e-05 - val_mean_absolute_error: 0.0067
Epoch 28/30
146/146 [==============================] - 4s 25ms/step - loss: 8.5208e-06 - mean_absolute_error: 0.0018 - val_loss: 6.3824e-05 - val_mean_absolute_error: 0.0067
Epoch 29/30
146/146 [==============================] - 4s 31ms/step - loss: 8.7315e-06 - mean_absolute_error: 0.0018 - val_loss: 6.1236e-05 - val_mean_absolute_error: 0.0065
Epoch 30/30
146/146 [==============================] - 4s 25ms/step - loss: 1.0987e-05 - mean_absolute_error: 0.0022 - val_loss: 6.6363e-05 - val_mean_absolute_error: 0.0066

# Let's plot the mean_squared_error and mean_absolute_error using model history on the entire dataset
plot_loss_and_mae_validation(
    model_name="model_128_1_30_3_months_validation",
    model_history=history_128_1_30_3_months_validation,
)

Observations

Using three months of data as a window to predict the stock price, the training loss and training mean absolute error are decreasing gradually, whereas the validation loss and validation mean absolute error are fluctuating, i.e., increasing and decreasing. The observed validation MAE is higher compared to that of using one LSTM layer with 64 neurons with 6 months of data as a window. One option to potentially further decrease the validation MAE is to increase or decrease the model's complexity by adding or reducing layers or neurons.

• 1 LSTM(128) 1 Dense(1) - 3 months
• Epochs: 30
• Batch Size: 16
• Training Loss: 1.0987e-05
• Validation Loss: 6.6363e-05
• Training MAE: 0.0022
• Validation MAE: 0.0065

Model Building with one LSTM layer and Dense layer (128-1) and 30 epochs - 6 months

# Let's create a keras sequential model
model_128_1_30_6_months_validation = Sequential()

# Let's add LSTM layer to the model network
model_128_1_30_6_months_validation.add(
    LSTM(128, return_sequences=False, input_shape=(x_train_6_months.shape[1], 1))
)
# Let's add dense layer to the model network
model_128_1_30_6_months_validation.add(Dense(1))

# Let's build the model using mean_squared_error as the loss function
# and mean_absolute_error as the evaluation metric during the compilation process
model_128_1_30_6_months_validation.compile(
    optimizer="adam", loss="mean_squared_error", metrics=["mean_absolute_error"]
)

# Let's define the callbacks for Model saving and Early stopping
cb_check = keras.callbacks.ModelCheckpoint(
    filepath="model_128_1_30_6_months_validation_checkpoint_filepath",
    save_best_only=True,
    monitor="val_loss",
)

cb_early = keras.callbacks.EarlyStopping(monitor="val_loss", patience=30)

# Let's fit the sequential model with input features and output label
history_128_1_30_6_months_validation = model_128_1_30_6_months_validation.fit(
    x_train_6_months,
    y_train_6_months,
    batch_size=16,
    epochs=30,
    callbacks=[cb_check, cb_early],
    validation_data=(x_valid_6_months, y_valid_6_months),
)

Epoch 1/30
140/140 [==============================] - 9s 57ms/step - loss: 1.2548e-04 - mean_absolute_error: 0.0056 - val_loss: 1.6437e-04 - val_mean_absolute_error: 0.0105
Epoch 2/30
140/140 [==============================] - 8s 55ms/step - loss: 2.6504e-05 - mean_absolute_error: 0.0034 - val_loss: 1.6220e-04 - val_mean_absolute_error: 0.0103
Epoch 3/30
140/140 [==============================] - 8s 57ms/step - loss: 2.1865e-05 - mean_absolute_error: 0.0031 - val_loss: 1.2300e-04 - val_mean_absolute_error: 0.0090
Epoch 4/30
140/140 [==============================] - 8s 57ms/step - loss: 2.0788e-05 - mean_absolute_error: 0.0028 - val_loss: 1.1704e-04 - val_mean_absolute_error: 0.0087
Epoch 5/30
140/140 [==============================] - 8s 57ms/step - loss: 1.9368e-05 - mean_absolute_error: 0.0028 - val_loss: 9.4113e-05 - val_mean_absolute_error: 0.0081
Epoch 6/30
140/140 [==============================] - 7s 48ms/step - loss: 1.6071e-05 - mean_absolute_error: 0.0025 - val_loss: 9.5063e-05 - val_mean_absolute_error: 0.0080
Epoch 7/30
140/140 [==============================] - 7s 49ms/step - loss: 1.8183e-05 - mean_absolute_error: 0.0028 - val_loss: 9.5268e-05 - val_mean_absolute_error: 0.0079
Epoch 8/30
140/140 [==============================] - 9s 66ms/step - loss: 1.7199e-05 - mean_absolute_error: 0.0027 - val_loss: 8.3562e-05 - val_mean_absolute_error: 0.0077
Epoch 9/30
140/140 [==============================] - 7s 49ms/step - loss: 1.7183e-05 - mean_absolute_error: 0.0027 - val_loss: 8.4873e-05 - val_mean_absolute_error: 0.0074
Epoch 10/30
140/140 [==============================] - 8s 56ms/step - loss: 1.5089e-05 - mean_absolute_error: 0.0025 - val_loss: 7.5727e-05 - val_mean_absolute_error: 0.0072
Epoch 11/30
140/140 [==============================] - 7s 48ms/step - loss: 1.4449e-05 - mean_absolute_error: 0.0024 - val_loss: 8.6800e-05 - val_mean_absolute_error: 0.0074
Epoch 12/30
140/140 [==============================] - 7s 48ms/step - loss: 1.2916e-05 - mean_absolute_error: 0.0023 - val_loss: 7.6817e-05 - val_mean_absolute_error: 0.0070
Epoch 13/30
140/140 [==============================] - 8s 56ms/step - loss: 1.1546e-05 - mean_absolute_error: 0.0021 - val_loss: 7.0710e-05 - val_mean_absolute_error: 0.0069
Epoch 14/30
140/140 [==============================] - 7s 48ms/step - loss: 1.1573e-05 - mean_absolute_error: 0.0021 - val_loss: 7.0749e-05 - val_mean_absolute_error: 0.0068
Epoch 15/30
140/140 [==============================] - 8s 54ms/step - loss: 1.2245e-05 - mean_absolute_error: 0.0023 - val_loss: 6.4123e-05 - val_mean_absolute_error: 0.0067
Epoch 16/30
140/140 [==============================] - 7s 49ms/step - loss: 1.3459e-05 - mean_absolute_error: 0.0024 - val_loss: 6.6394e-05 - val_mean_absolute_error: 0.0070
Epoch 17/30
140/140 [==============================] - 8s 55ms/step - loss: 1.2017e-05 - mean_absolute_error: 0.0023 - val_loss: 6.0408e-05 - val_mean_absolute_error: 0.0066
Epoch 18/30
140/140 [==============================] - 7s 48ms/step - loss: 1.0485e-05 - mean_absolute_error: 0.0020 - val_loss: 6.3196e-05 - val_mean_absolute_error: 0.0065
Epoch 19/30
140/140 [==============================] - 7s 47ms/step - loss: 1.1530e-05 - mean_absolute_error: 0.0022 - val_loss: 6.4957e-05 - val_mean_absolute_error: 0.0065
Epoch 20/30
140/140 [==============================] - 7s 48ms/step - loss: 1.0393e-05 - mean_absolute_error: 0.0020 - val_loss: 6.1495e-05 - val_mean_absolute_error: 0.0064
Epoch 21/30
140/140 [==============================] - 8s 59ms/step - loss: 1.0281e-05 - mean_absolute_error: 0.0021 - val_loss: 5.9537e-05 - val_mean_absolute_error: 0.0063
Epoch 22/30
140/140 [==============================] - 8s 56ms/step - loss: 8.7386e-06 - mean_absolute_error: 0.0018 - val_loss: 5.5928e-05 - val_mean_absolute_error: 0.0062
Epoch 23/30
140/140 [==============================] - 7s 50ms/step - loss: 9.7694e-06 - mean_absolute_error: 0.0019 - val_loss: 5.6893e-05 - val_mean_absolute_error: 0.0065
Epoch 24/30
140/140 [==============================] - 9s 61ms/step - loss: 9.7910e-06 - mean_absolute_error: 0.0020 - val_loss: 5.4273e-05 - val_mean_absolute_error: 0.0062
Epoch 25/30
140/140 [==============================] - 7s 48ms/step - loss: 8.6159e-06 - mean_absolute_error: 0.0017 - val_loss: 9.7429e-05 - val_mean_absolute_error: 0.0080
Epoch 26/30
140/140 [==============================] - 8s 56ms/step - loss: 9.5577e-06 - mean_absolute_error: 0.0020 - val_loss: 5.3570e-05 - val_mean_absolute_error: 0.0061
Epoch 27/30
140/140 [==============================] - 8s 55ms/step - loss: 1.4297e-05 - mean_absolute_error: 0.0025 - val_loss: 5.2298e-05 - val_mean_absolute_error: 0.0059
Epoch 28/30
140/140 [==============================] - 7s 49ms/step - loss: 9.3608e-06 - mean_absolute_error: 0.0019 - val_loss: 6.2286e-05 - val_mean_absolute_error: 0.0062
Epoch 29/30
140/140 [==============================] - 7s 48ms/step - loss: 8.9242e-06 - mean_absolute_error: 0.0019 - val_loss: 6.3262e-05 - val_mean_absolute_error: 0.0063
Epoch 30/30
140/140 [==============================] - 7s 49ms/step - loss: 8.3871e-06 - mean_absolute_error: 0.0018 - val_loss: 5.6586e-05 - val_mean_absolute_error: 0.0062

# Let's plot the mean_squared_error and mean_absolute_error using model history on the entire dataset
plot_loss_and_mae_validation(
    model_name="model_128_1_30_6_months_validation",
    model_history=history_128_1_30_6_months_validation,
)

Observations

Using six months of data as a window to predict the stock price, the training loss and training mean absolute error are decreasing gradually, whereas the validation loss and validation mean absolute error are fluctuating, i.e., increasing and decreasing. The observed validation MAE is similar to that of compared to that of using one LSTM layer with 64 neurons with 6 months of data as a window. One option to potentially further decrease the validation MAE is to increase or decrease the model's complexity by adding or reducing layers or neurons.

• 1 LSTM(128) 1 Dense(1) - 6 months
• Epochs: 30
• Batch Size: 16
• Training Loss: 8.3871e-06
• Validation Loss: 5.6586e-05
• Training MAE: 0.0018
• Validation MAE: 0.0062

Model Building with 2 LSTM layers and two Dense layers (128-64-8-1) and 30 epochs - 3 months

# Let's create a keras sequential model
model_128_64_8_1_30_3_months_validation = Sequential()

# Let's add two LSTM layer's to the model network
model_128_64_8_1_30_3_months_validation.add(
    LSTM(128, return_sequences=True, input_shape=(x_train_3_months.shape[1], 1))
)
model_128_64_8_1_30_3_months_validation.add(LSTM(64, return_sequences=False))

# Let's add dense layer to the model network
model_128_64_8_1_30_3_months_validation.add(Dense(8))
model_128_64_8_1_30_3_months_validation.add(Dense(1))

# Let's build the model using mean_squared_error as the loss function
# and mean_absolute_error as the evaluation metric during the compilation process
model_128_64_8_1_30_3_months_validation.compile(
    optimizer="adam", loss="mean_squared_error", metrics=["mean_absolute_error"]
)

# Let's define the callbacks for Model saving and Early stopping
cb_check = keras.callbacks.ModelCheckpoint(
    filepath="model_128_64_8_1_30_3_months_validation_checkpoint_filepath",
    save_best_only=True,
    monitor="val_loss",
)

cb_early = keras.callbacks.EarlyStopping(monitor="val_loss", patience=30)

# Let's fit the sequential model with input features and output label
history_128_64_8_1_30_3_months_validation = model_128_64_8_1_30_3_months_validation.fit(
    x_train_3_months,
    y_train_3_months,
    batch_size=16,
    epochs=30,
    callbacks=[cb_check, cb_early],
    validation_data=(x_valid_3_months, y_valid_3_months),
)

Epoch 1/30
146/146 [==============================] - 10s 60ms/step - loss: 1.5865e-04 - mean_absolute_error: 0.0061 - val_loss: 3.4999e-04 - val_mean_absolute_error: 0.0147
Epoch 2/30
146/146 [==============================] - 8s 58ms/step - loss: 4.0777e-05 - mean_absolute_error: 0.0043 - val_loss: 1.8920e-04 - val_mean_absolute_error: 0.0113
Epoch 3/30
146/146 [==============================] - 9s 59ms/step - loss: 3.1651e-05 - mean_absolute_error: 0.0037 - val_loss: 1.5898e-04 - val_mean_absolute_error: 0.0103
Epoch 4/30
146/146 [==============================] - 7s 49ms/step - loss: 3.3430e-05 - mean_absolute_error: 0.0041 - val_loss: 1.5997e-04 - val_mean_absolute_error: 0.0103
Epoch 5/30
146/146 [==============================] - 9s 59ms/step - loss: 5.2252e-05 - mean_absolute_error: 0.0048 - val_loss: 1.4264e-04 - val_mean_absolute_error: 0.0096
Epoch 6/30
146/146 [==============================] - 11s 72ms/step - loss: 2.0082e-05 - mean_absolute_error: 0.0029 - val_loss: 1.2257e-04 - val_mean_absolute_error: 0.0089
Epoch 7/30
146/146 [==============================] - 7s 49ms/step - loss: 1.7257e-05 - mean_absolute_error: 0.0026 - val_loss: 1.7720e-04 - val_mean_absolute_error: 0.0104
Epoch 8/30
146/146 [==============================] - 7s 49ms/step - loss: 2.6818e-05 - mean_absolute_error: 0.0034 - val_loss: 1.5267e-04 - val_mean_absolute_error: 0.0102
Epoch 9/30
146/146 [==============================] - 9s 60ms/step - loss: 1.5620e-05 - mean_absolute_error: 0.0025 - val_loss: 1.1458e-04 - val_mean_absolute_error: 0.0085
Epoch 10/30
146/146 [==============================] - 7s 46ms/step - loss: 1.6709e-05 - mean_absolute_error: 0.0026 - val_loss: 1.5275e-04 - val_mean_absolute_error: 0.0096
Epoch 11/30
146/146 [==============================] - 7s 46ms/step - loss: 1.3953e-05 - mean_absolute_error: 0.0024 - val_loss: 1.3259e-04 - val_mean_absolute_error: 0.0089
Epoch 12/30
146/146 [==============================] - 7s 49ms/step - loss: 2.0151e-05 - mean_absolute_error: 0.0031 - val_loss: 1.1610e-04 - val_mean_absolute_error: 0.0084
Epoch 13/30
146/146 [==============================] - 9s 60ms/step - loss: 1.7692e-05 - mean_absolute_error: 0.0029 - val_loss: 1.0597e-04 - val_mean_absolute_error: 0.0082
Epoch 14/30
146/146 [==============================] - 9s 63ms/step - loss: 1.4083e-05 - mean_absolute_error: 0.0025 - val_loss: 8.9251e-05 - val_mean_absolute_error: 0.0077
Epoch 15/30
146/146 [==============================] - 8s 51ms/step - loss: 1.3823e-05 - mean_absolute_error: 0.0026 - val_loss: 2.5661e-04 - val_mean_absolute_error: 0.0131
Epoch 16/30
146/146 [==============================] - 9s 60ms/step - loss: 2.2210e-05 - mean_absolute_error: 0.0034 - val_loss: 8.1642e-05 - val_mean_absolute_error: 0.0076
Epoch 17/30
146/146 [==============================] - 7s 49ms/step - loss: 1.1925e-05 - mean_absolute_error: 0.0021 - val_loss: 1.1677e-04 - val_mean_absolute_error: 0.0083
Epoch 18/30
146/146 [==============================] - 7s 48ms/step - loss: 1.2013e-05 - mean_absolute_error: 0.0022 - val_loss: 1.1438e-04 - val_mean_absolute_error: 0.0088
Epoch 19/30
146/146 [==============================] - 7s 49ms/step - loss: 1.2903e-05 - mean_absolute_error: 0.0025 - val_loss: 1.7209e-04 - val_mean_absolute_error: 0.0104
Epoch 20/30
146/146 [==============================] - 8s 53ms/step - loss: 1.3561e-05 - mean_absolute_error: 0.0023 - val_loss: 9.6123e-05 - val_mean_absolute_error: 0.0076
Epoch 21/30
146/146 [==============================] - 8s 53ms/step - loss: 1.1935e-05 - mean_absolute_error: 0.0023 - val_loss: 9.7677e-05 - val_mean_absolute_error: 0.0081
Epoch 22/30
146/146 [==============================] - 7s 51ms/step - loss: 1.3632e-05 - mean_absolute_error: 0.0025 - val_loss: 9.1015e-05 - val_mean_absolute_error: 0.0075
Epoch 23/30
146/146 [==============================] - 7s 51ms/step - loss: 1.4661e-05 - mean_absolute_error: 0.0026 - val_loss: 1.0996e-04 - val_mean_absolute_error: 0.0082
Epoch 24/30
146/146 [==============================] - 11s 75ms/step - loss: 1.1475e-05 - mean_absolute_error: 0.0023 - val_loss: 6.5836e-05 - val_mean_absolute_error: 0.0067
Epoch 25/30
146/146 [==============================] - 9s 59ms/step - loss: 1.4135e-05 - mean_absolute_error: 0.0025 - val_loss: 6.4055e-05 - val_mean_absolute_error: 0.0066
Epoch 26/30
146/146 [==============================] - 7s 49ms/step - loss: 1.1557e-05 - mean_absolute_error: 0.0022 - val_loss: 6.5134e-05 - val_mean_absolute_error: 0.0065
Epoch 27/30
146/146 [==============================] - 7s 50ms/step - loss: 1.2161e-05 - mean_absolute_error: 0.0024 - val_loss: 7.6511e-05 - val_mean_absolute_error: 0.0072
Epoch 28/30
146/146 [==============================] - 9s 62ms/step - loss: 9.1096e-06 - mean_absolute_error: 0.0019 - val_loss: 5.8593e-05 - val_mean_absolute_error: 0.0063
Epoch 29/30
146/146 [==============================] - 7s 49ms/step - loss: 1.6023e-05 - mean_absolute_error: 0.0028 - val_loss: 7.1138e-05 - val_mean_absolute_error: 0.0067
Epoch 30/30
146/146 [==============================] - 7s 49ms/step - loss: 7.3708e-06 - mean_absolute_error: 0.0017 - val_loss: 1.0489e-04 - val_mean_absolute_error: 0.0083

# Let's plot the mean_squared_error and mean_absolute_error using model history on the entire dataset
plot_loss_and_mae_validation(
    model_name="model_128_64_8_1_30_3_months_validation",
    model_history=history_128_64_8_1_30_3_months_validation,
)

Observations

Using three months of data as a window to predict the stock price, the training loss and training mean absolute error are decreasing gradually, whereas the validation loss and validation MAE are fluctuating, i.e., increasing and decreasing. The observed validation MAE is higher to that of compared to that of using one LSTM layer with 128 neurons with 6 months of data as a window. One option to potentially decrease the validation MAE further is to increase or decrease the model's complexity.

• 2 LSTM(128-64) 2 Dense(8-1) - 3 months
• Epochs: 30
• Batch Size: 16
• Training Loss: 7.3708e-06
• Validation Loss: 1.0489e-04
• Training MAE: 0.0017
• Validation MAE: 0.0083

Model Building with 2 LSTM layers and two Dense layers (128-64-8-1) and 30 epochs - 6 months

# Let's create a keras sequential model
model_128_64_8_1_30_6_months_validation = Sequential()

# Let's add two LSTM layer's to the model network
model_128_64_8_1_30_6_months_validation.add(
    LSTM(128, return_sequences=True, input_shape=(x_train_6_months.shape[1], 1))
)
model_128_64_8_1_30_6_months_validation.add(LSTM(64, return_sequences=False))

# Let's add dense layer to the model network
model_128_64_8_1_30_6_months_validation.add(Dense(8))
model_128_64_8_1_30_6_months_validation.add(Dense(1))

# Let's build the model using mean_squared_error as the loss function
# and mean_absolute_error as the evaluation metric during the compilation process
model_128_64_8_1_30_6_months_validation.compile(
    optimizer="adam", loss="mean_squared_error", metrics=["mean_absolute_error"]
)

# Let's define the callbacks for Model saving and Early stopping
cb_check = keras.callbacks.ModelCheckpoint(
    filepath="model_128_64_8_1_30_6_months_validation_checkpoint_filepath",
    save_best_only=True,
    monitor="val_loss",
)

cb_early = keras.callbacks.EarlyStopping(monitor="val_loss", patience=30)

# Let's fit the sequential model with input features and output label
history_128_64_8_1_30_6_months_validation = model_128_64_8_1_30_6_months_validation.fit(
    x_train_6_months,
    y_train_6_months,
    batch_size=16,
    epochs=30,
    callbacks=[cb_check, cb_early],
    validation_data=(x_valid_6_months, y_valid_6_months),
)

Epoch 1/30
140/140 [==============================] - 16s 105ms/step - loss: 2.3243e-04 - mean_absolute_error: 0.0071 - val_loss: 3.1130e-04 - val_mean_absolute_error: 0.0138
Epoch 2/30
140/140 [==============================] - 16s 114ms/step - loss: 4.1988e-05 - mean_absolute_error: 0.0042 - val_loss: 2.2554e-04 - val_mean_absolute_error: 0.0121
Epoch 3/30
140/140 [==============================] - 12s 85ms/step - loss: 3.1558e-05 - mean_absolute_error: 0.0036 - val_loss: 2.2600e-04 - val_mean_absolute_error: 0.0123
Epoch 4/30
140/140 [==============================] - 14s 101ms/step - loss: 2.6691e-05 - mean_absolute_error: 0.0033 - val_loss: 2.0955e-04 - val_mean_absolute_error: 0.0113
Epoch 5/30
140/140 [==============================] - 14s 99ms/step - loss: 2.4613e-05 - mean_absolute_error: 0.0034 - val_loss: 1.3687e-04 - val_mean_absolute_error: 0.0095
Epoch 6/30
140/140 [==============================] - 12s 86ms/step - loss: 2.3111e-05 - mean_absolute_error: 0.0032 - val_loss: 1.3792e-04 - val_mean_absolute_error: 0.0093
Epoch 7/30
140/140 [==============================] - 14s 99ms/step - loss: 2.4380e-05 - mean_absolute_error: 0.0033 - val_loss: 1.0501e-04 - val_mean_absolute_error: 0.0086
Epoch 8/30
140/140 [==============================] - 14s 99ms/step - loss: 1.8261e-05 - mean_absolute_error: 0.0026 - val_loss: 9.5749e-05 - val_mean_absolute_error: 0.0081
Epoch 9/30
140/140 [==============================] - 12s 88ms/step - loss: 1.8566e-05 - mean_absolute_error: 0.0029 - val_loss: 1.0044e-04 - val_mean_absolute_error: 0.0083
Epoch 10/30
140/140 [==============================] - 12s 87ms/step - loss: 1.7971e-05 - mean_absolute_error: 0.0027 - val_loss: 1.1114e-04 - val_mean_absolute_error: 0.0085
Epoch 11/30
140/140 [==============================] - 14s 101ms/step - loss: 1.6917e-05 - mean_absolute_error: 0.0027 - val_loss: 8.7027e-05 - val_mean_absolute_error: 0.0077
Epoch 12/30
140/140 [==============================] - 16s 114ms/step - loss: 1.8881e-05 - mean_absolute_error: 0.0030 - val_loss: 8.2377e-05 - val_mean_absolute_error: 0.0077
Epoch 13/30
140/140 [==============================] - 14s 98ms/step - loss: 1.5331e-05 - mean_absolute_error: 0.0025 - val_loss: 7.6972e-05 - val_mean_absolute_error: 0.0075
Epoch 14/30
140/140 [==============================] - 12s 87ms/step - loss: 1.8943e-05 - mean_absolute_error: 0.0029 - val_loss: 8.1627e-05 - val_mean_absolute_error: 0.0077
Epoch 15/30
140/140 [==============================] - 12s 86ms/step - loss: 2.1172e-05 - mean_absolute_error: 0.0033 - val_loss: 1.6213e-04 - val_mean_absolute_error: 0.0105
Epoch 16/30
140/140 [==============================] - 12s 86ms/step - loss: 1.9446e-05 - mean_absolute_error: 0.0029 - val_loss: 1.1488e-04 - val_mean_absolute_error: 0.0084
Epoch 17/30
140/140 [==============================] - 14s 98ms/step - loss: 1.3487e-05 - mean_absolute_error: 0.0024 - val_loss: 6.7038e-05 - val_mean_absolute_error: 0.0071
Epoch 18/30
140/140 [==============================] - 12s 86ms/step - loss: 1.5220e-05 - mean_absolute_error: 0.0026 - val_loss: 7.3298e-05 - val_mean_absolute_error: 0.0073
Epoch 19/30
140/140 [==============================] - 14s 98ms/step - loss: 1.3266e-05 - mean_absolute_error: 0.0024 - val_loss: 6.6475e-05 - val_mean_absolute_error: 0.0070
Epoch 20/30
140/140 [==============================] - 12s 87ms/step - loss: 1.2591e-05 - mean_absolute_error: 0.0024 - val_loss: 8.7832e-05 - val_mean_absolute_error: 0.0078
Epoch 21/30
140/140 [==============================] - 12s 87ms/step - loss: 1.4575e-05 - mean_absolute_error: 0.0026 - val_loss: 1.1000e-04 - val_mean_absolute_error: 0.0084
Epoch 22/30
140/140 [==============================] - 14s 100ms/step - loss: 1.5419e-05 - mean_absolute_error: 0.0026 - val_loss: 6.1326e-05 - val_mean_absolute_error: 0.0065
Epoch 23/30
140/140 [==============================] - 12s 87ms/step - loss: 1.3730e-05 - mean_absolute_error: 0.0026 - val_loss: 6.3581e-05 - val_mean_absolute_error: 0.0067
Epoch 24/30
140/140 [==============================] - 12s 87ms/step - loss: 1.1162e-05 - mean_absolute_error: 0.0022 - val_loss: 6.5046e-05 - val_mean_absolute_error: 0.0067
Epoch 25/30
140/140 [==============================] - 12s 87ms/step - loss: 1.3816e-05 - mean_absolute_error: 0.0025 - val_loss: 9.7000e-05 - val_mean_absolute_error: 0.0081
Epoch 26/30
140/140 [==============================] - 12s 85ms/step - loss: 1.3579e-05 - mean_absolute_error: 0.0026 - val_loss: 1.2669e-04 - val_mean_absolute_error: 0.0093
Epoch 27/30
140/140 [==============================] - 12s 86ms/step - loss: 1.3376e-05 - mean_absolute_error: 0.0026 - val_loss: 1.0260e-04 - val_mean_absolute_error: 0.0081
Epoch 28/30
140/140 [==============================] - 15s 110ms/step - loss: 1.2869e-05 - mean_absolute_error: 0.0025 - val_loss: 5.7025e-05 - val_mean_absolute_error: 0.0061
Epoch 29/30
140/140 [==============================] - 12s 86ms/step - loss: 9.8393e-06 - mean_absolute_error: 0.0021 - val_loss: 1.0253e-04 - val_mean_absolute_error: 0.0084
Epoch 30/30
140/140 [==============================] - 12s 86ms/step - loss: 1.6302e-05 - mean_absolute_error: 0.0028 - val_loss: 1.3228e-04 - val_mean_absolute_error: 0.0096

# Let's plot the mean_squared_error and mean_absolute_error using model history on the entire dataset
plot_loss_and_mae_validation(
    model_name="model_128_64_8_1_30_6_months_validation",
    model_history=history_128_64_8_1_30_6_months_validation,
)

Observations

Using six months of data as a window to predict the stock price, the training loss and training mean absolute error are decreasing gradually, whereas the validation loss and validation MAE are fluctuating, i.e., increasing and decreasing. The observed validation MAE is almost similar to that of compared to that of using one LSTM layer with 64 neurons with 6 months of data as a window. One option to potentially decrease the validation MAE further is to increase or decrease the model's complexity.

• 2 LSTM(128-64) 2 Dense(8-1) - 6 months
• Epochs: 30
• Batch Size: 16
• Training Loss: 1.6302e-05
• Validation Loss: 1.3228e-04
• Training MAE: 0.0028
• Validation MAE: 0.0061

Model Building with 3 LSTM and 3 Dense layers (128-64-64-8-8-1) and 30 epochs - 3 months

# Let's create a keras sequential model
model_128_64_64_8_8_1_30_3_months_validation = Sequential()

# Let's add two LSTM layer's to the model network
model_128_64_64_8_8_1_30_3_months_validation.add(
    LSTM(128, return_sequences=True, input_shape=(x_train_3_months.shape[1], 1))
)
model_128_64_64_8_8_1_30_3_months_validation.add(LSTM(64, return_sequences=True))
model_128_64_64_8_8_1_30_3_months_validation.add(LSTM(64, return_sequences=False))

# Let's add dense layer to the model network
model_128_64_64_8_8_1_30_3_months_validation.add(Dense(8))
model_128_64_64_8_8_1_30_3_months_validation.add(Dense(8))
model_128_64_64_8_8_1_30_3_months_validation.add(Dense(1))

# Let's build the model using mean_squared_error as the loss function
# and mean_absolute_error as the evaluation metric during the compilation process
model_128_64_64_8_8_1_30_3_months_validation.compile(
    optimizer="adam", loss="mean_squared_error", metrics=["mean_absolute_error"]
)

# Let's define the callbacks for Model saving and Early stopping
cb_check = keras.callbacks.ModelCheckpoint(
    filepath="model_128_64_64_8_8_1_30_3_months_validation_checkpoint_filepath",
    save_best_only=True,
    monitor="val_loss",
)

cb_early = keras.callbacks.EarlyStopping(monitor="val_loss", patience=30)

# Let's fit the sequential model with input features and output label
history_128_64_64_8_8_1_30_3_months_validation = (
    model_128_64_64_8_8_1_30_3_months_validation.fit(
        x_train_3_months,
        y_train_3_months,
        batch_size=16,
        epochs=30,
        callbacks=[cb_check, cb_early],
        validation_data=(x_valid_3_months, y_valid_3_months),
    )
)

Epoch 1/30
146/146 [==============================] - 14s 85ms/step - loss: 2.3189e-04 - mean_absolute_error: 0.0083 - val_loss: 6.6465e-04 - val_mean_absolute_error: 0.0205
Epoch 2/30
146/146 [==============================] - 9s 63ms/step - loss: 6.7811e-05 - mean_absolute_error: 0.0052 - val_loss: 7.0893e-04 - val_mean_absolute_error: 0.0210
Epoch 3/30
146/146 [==============================] - 12s 81ms/step - loss: 7.1001e-05 - mean_absolute_error: 0.0059 - val_loss: 5.0707e-04 - val_mean_absolute_error: 0.0183
Epoch 4/30
146/146 [==============================] - 13s 89ms/step - loss: 3.9086e-05 - mean_absolute_error: 0.0040 - val_loss: 2.6350e-04 - val_mean_absolute_error: 0.0135
Epoch 5/30
146/146 [==============================] - 9s 60ms/step - loss: 4.2576e-05 - mean_absolute_error: 0.0045 - val_loss: 3.1700e-04 - val_mean_absolute_error: 0.0141
Epoch 6/30
146/146 [==============================] - 12s 80ms/step - loss: 3.4198e-05 - mean_absolute_error: 0.0040 - val_loss: 1.9118e-04 - val_mean_absolute_error: 0.0113
Epoch 7/30
146/146 [==============================] - 9s 63ms/step - loss: 3.0268e-05 - mean_absolute_error: 0.0038 - val_loss: 2.6514e-04 - val_mean_absolute_error: 0.0128
Epoch 8/30
146/146 [==============================] - 12s 80ms/step - loss: 3.2691e-05 - mean_absolute_error: 0.0040 - val_loss: 1.7674e-04 - val_mean_absolute_error: 0.0105
Epoch 9/30
146/146 [==============================] - 9s 63ms/step - loss: 3.2308e-05 - mean_absolute_error: 0.0037 - val_loss: 2.0273e-04 - val_mean_absolute_error: 0.0111
Epoch 10/30
146/146 [==============================] - 10s 70ms/step - loss: 4.0137e-05 - mean_absolute_error: 0.0044 - val_loss: 1.9199e-04 - val_mean_absolute_error: 0.0108
Epoch 11/30
146/146 [==============================] - 12s 81ms/step - loss: 2.3449e-05 - mean_absolute_error: 0.0032 - val_loss: 1.3262e-04 - val_mean_absolute_error: 0.0094
Epoch 12/30
146/146 [==============================] - 15s 104ms/step - loss: 2.4884e-05 - mean_absolute_error: 0.0032 - val_loss: 1.2520e-04 - val_mean_absolute_error: 0.0092
Epoch 13/30
146/146 [==============================] - 11s 78ms/step - loss: 1.8455e-05 - mean_absolute_error: 0.0028 - val_loss: 1.2194e-04 - val_mean_absolute_error: 0.0091
Epoch 14/30
146/146 [==============================] - 11s 77ms/step - loss: 1.7568e-05 - mean_absolute_error: 0.0028 - val_loss: 1.2041e-04 - val_mean_absolute_error: 0.0091
Epoch 15/30
146/146 [==============================] - 9s 60ms/step - loss: 3.2445e-05 - mean_absolute_error: 0.0037 - val_loss: 1.6187e-04 - val_mean_absolute_error: 0.0105
Epoch 16/30
146/146 [==============================] - 9s 61ms/step - loss: 3.4384e-05 - mean_absolute_error: 0.0040 - val_loss: 1.7437e-04 - val_mean_absolute_error: 0.0103
Epoch 17/30
146/146 [==============================] - 11s 77ms/step - loss: 1.8467e-05 - mean_absolute_error: 0.0030 - val_loss: 1.0155e-04 - val_mean_absolute_error: 0.0084
Epoch 18/30
146/146 [==============================] - 9s 60ms/step - loss: 2.1996e-05 - mean_absolute_error: 0.0033 - val_loss: 1.3253e-04 - val_mean_absolute_error: 0.0090
Epoch 19/30
146/146 [==============================] - 9s 60ms/step - loss: 2.5423e-05 - mean_absolute_error: 0.0036 - val_loss: 1.0796e-04 - val_mean_absolute_error: 0.0084
Epoch 20/30
146/146 [==============================] - 9s 61ms/step - loss: 1.5167e-05 - mean_absolute_error: 0.0026 - val_loss: 1.2306e-04 - val_mean_absolute_error: 0.0087
Epoch 21/30
146/146 [==============================] - 9s 59ms/step - loss: 2.3926e-05 - mean_absolute_error: 0.0034 - val_loss: 1.0201e-04 - val_mean_absolute_error: 0.0081
Epoch 22/30
146/146 [==============================] - 9s 59ms/step - loss: 1.3370e-05 - mean_absolute_error: 0.0024 - val_loss: 1.3651e-04 - val_mean_absolute_error: 0.0091
Epoch 23/30
146/146 [==============================] - 13s 86ms/step - loss: 1.3756e-05 - mean_absolute_error: 0.0025 - val_loss: 8.7949e-05 - val_mean_absolute_error: 0.0076
Epoch 24/30
146/146 [==============================] - 11s 78ms/step - loss: 1.6606e-05 - mean_absolute_error: 0.0028 - val_loss: 8.6171e-05 - val_mean_absolute_error: 0.0075
Epoch 25/30
146/146 [==============================] - 11s 77ms/step - loss: 2.1919e-05 - mean_absolute_error: 0.0034 - val_loss: 8.3205e-05 - val_mean_absolute_error: 0.0074
Epoch 26/30
146/146 [==============================] - 9s 59ms/step - loss: 1.8715e-05 - mean_absolute_error: 0.0029 - val_loss: 1.3098e-04 - val_mean_absolute_error: 0.0090
Epoch 27/30
146/146 [==============================] - 8s 58ms/step - loss: 1.2661e-05 - mean_absolute_error: 0.0024 - val_loss: 1.3720e-04 - val_mean_absolute_error: 0.0093
Epoch 28/30
146/146 [==============================] - 11s 77ms/step - loss: 1.2176e-05 - mean_absolute_error: 0.0023 - val_loss: 6.7984e-05 - val_mean_absolute_error: 0.0068
Epoch 29/30
146/146 [==============================] - 9s 59ms/step - loss: 1.8711e-05 - mean_absolute_error: 0.0030 - val_loss: 8.2676e-05 - val_mean_absolute_error: 0.0072
Epoch 30/30
146/146 [==============================] - 9s 59ms/step - loss: 2.0030e-05 - mean_absolute_error: 0.0029 - val_loss: 9.0370e-05 - val_mean_absolute_error: 0.0075

# Let's plot the mean_squared_error and mean_absolute_error using model history on the entire dataset
plot_loss_and_mae_validation(
    model_name="model_128_64_64_8_8_1_30_3_months_validation",
    model_history=history_128_64_64_8_8_1_30_3_months_validation,
)

Observations

Using three months of data as a window to predict the stock price, the training loss and training mean absolute error are decreasing gradually, whereas the validation loss and validation MAE are fluctuating, i.e., increasing and decreasing. The observed validation MAE is almost similar to that of compared to that of using 2 LSTM layers and 2 Dense layers with 6 months of data as a window. One option to potentially decrease the validation MAE further is to increase or decrease the model's complexity.

• 3 LSTM(128-64-64) 3 Dense(8-8-1) - 3 months
• Epochs: 30
• Batch Size: 16
• Training Loss: 2.0030e-05
• Validation Loss: 9.0370e-05
• Training MAE: 0.0023
• Validation MAE: 0.0068

Model Building with 3 LSTM and 3 Dense layers (128-64-64-8-8-1) and 30 epochs - 6 months

# Let's create a keras sequential model
model_128_64_64_8_8_1_30_6_months_validation = Sequential()

# Let's add two LSTM layer's to the model network
model_128_64_64_8_8_1_30_6_months_validation.add(
    LSTM(128, return_sequences=True, input_shape=(x_train_6_months.shape[1], 1))
)
model_128_64_64_8_8_1_30_6_months_validation.add(LSTM(64, return_sequences=True))
model_128_64_64_8_8_1_30_6_months_validation.add(LSTM(64, return_sequences=False))

# Let's add dense layer to the model network
model_128_64_64_8_8_1_30_6_months_validation.add(Dense(8))
model_128_64_64_8_8_1_30_6_months_validation.add(Dense(8))
model_128_64_64_8_8_1_30_6_months_validation.add(Dense(1))

# Let's build the model using mean_squared_error as the loss function
# and mean_absolute_error as the evaluation metric during the compilation process
model_128_64_64_8_8_1_30_6_months_validation.compile(
    optimizer="adam", loss="mean_squared_error", metrics=["mean_absolute_error"]
)

# Let's define the callbacks for Model saving and Early stopping
cb_check = keras.callbacks.ModelCheckpoint(
    filepath="model_128_64_64_8_8_1_30_6_months_validation_checkpoint_filepath",
    save_best_only=True,
    monitor="val_loss",
)

cb_early = keras.callbacks.EarlyStopping(monitor="val_loss", patience=30)

# Let's fit the sequential model with input features and output label
history_128_64_64_8_8_1_30_6_months_validation = (
    model_128_64_64_8_8_1_30_6_months_validation.fit(
        x_train_6_months,
        y_train_6_months,
        batch_size=16,
        epochs=30,
        callbacks=[cb_check, cb_early],
        validation_data=(x_valid_6_months, y_valid_6_months),
    )
)

Epoch 1/30
140/140 [==============================] - 23s 136ms/step - loss: 1.8734e-04 - mean_absolute_error: 0.0077 - val_loss: 3.8976e-04 - val_mean_absolute_error: 0.0166
Epoch 2/30
140/140 [==============================] - 16s 115ms/step - loss: 5.6671e-05 - mean_absolute_error: 0.0050 - val_loss: 6.7751e-04 - val_mean_absolute_error: 0.0217
Epoch 3/30
140/140 [==============================] - 19s 136ms/step - loss: 4.2503e-05 - mean_absolute_error: 0.0041 - val_loss: 2.6947e-04 - val_mean_absolute_error: 0.0135
Epoch 4/30
140/140 [==============================] - 16s 117ms/step - loss: 3.8298e-05 - mean_absolute_error: 0.0041 - val_loss: 3.6633e-04 - val_mean_absolute_error: 0.0150
Epoch 5/30
140/140 [==============================] - 19s 136ms/step - loss: 4.9774e-05 - mean_absolute_error: 0.0049 - val_loss: 2.6623e-04 - val_mean_absolute_error: 0.0134
Epoch 6/30
140/140 [==============================] - 20s 142ms/step - loss: 2.5409e-05 - mean_absolute_error: 0.0032 - val_loss: 1.6868e-04 - val_mean_absolute_error: 0.0104
Epoch 7/30
140/140 [==============================] - 16s 115ms/step - loss: 2.9503e-05 - mean_absolute_error: 0.0036 - val_loss: 2.8543e-04 - val_mean_absolute_error: 0.0136
Epoch 8/30
140/140 [==============================] - 19s 133ms/step - loss: 3.4069e-05 - mean_absolute_error: 0.0040 - val_loss: 1.2818e-04 - val_mean_absolute_error: 0.0093
Epoch 9/30
140/140 [==============================] - 16s 117ms/step - loss: 2.5651e-05 - mean_absolute_error: 0.0033 - val_loss: 1.3200e-04 - val_mean_absolute_error: 0.0096
Epoch 10/30
140/140 [==============================] - 16s 115ms/step - loss: 3.0092e-05 - mean_absolute_error: 0.0038 - val_loss: 1.4585e-04 - val_mean_absolute_error: 0.0095
Epoch 11/30
140/140 [==============================] - 19s 134ms/step - loss: 2.8157e-05 - mean_absolute_error: 0.0038 - val_loss: 1.1037e-04 - val_mean_absolute_error: 0.0089
Epoch 12/30
140/140 [==============================] - 16s 115ms/step - loss: 2.2718e-05 - mean_absolute_error: 0.0033 - val_loss: 1.1270e-04 - val_mean_absolute_error: 0.0086
Epoch 13/30
140/140 [==============================] - 16s 115ms/step - loss: 2.1889e-05 - mean_absolute_error: 0.0032 - val_loss: 1.1763e-04 - val_mean_absolute_error: 0.0091
Epoch 14/30
140/140 [==============================] - 16s 115ms/step - loss: 2.6418e-05 - mean_absolute_error: 0.0037 - val_loss: 4.4169e-04 - val_mean_absolute_error: 0.0187
Epoch 15/30
140/140 [==============================] - 19s 137ms/step - loss: 3.8126e-05 - mean_absolute_error: 0.0040 - val_loss: 8.8671e-05 - val_mean_absolute_error: 0.0079
Epoch 16/30
140/140 [==============================] - 18s 129ms/step - loss: 2.5671e-05 - mean_absolute_error: 0.0035 - val_loss: 9.2030e-05 - val_mean_absolute_error: 0.0078
Epoch 17/30
140/140 [==============================] - 19s 134ms/step - loss: 2.0304e-05 - mean_absolute_error: 0.0030 - val_loss: 8.4757e-05 - val_mean_absolute_error: 0.0076
Epoch 18/30
140/140 [==============================] - 19s 137ms/step - loss: 1.8308e-05 - mean_absolute_error: 0.0029 - val_loss: 7.2501e-05 - val_mean_absolute_error: 0.0072
Epoch 19/30
140/140 [==============================] - 16s 115ms/step - loss: 1.4187e-05 - mean_absolute_error: 0.0024 - val_loss: 1.8342e-04 - val_mean_absolute_error: 0.0111
Epoch 20/30
140/140 [==============================] - 16s 116ms/step - loss: 2.0199e-05 - mean_absolute_error: 0.0029 - val_loss: 1.0401e-04 - val_mean_absolute_error: 0.0080
Epoch 21/30
140/140 [==============================] - 16s 117ms/step - loss: 1.5210e-05 - mean_absolute_error: 0.0026 - val_loss: 3.9336e-04 - val_mean_absolute_error: 0.0177
Epoch 22/30
140/140 [==============================] - 16s 116ms/step - loss: 2.9060e-05 - mean_absolute_error: 0.0037 - val_loss: 1.3103e-04 - val_mean_absolute_error: 0.0092
Epoch 23/30
140/140 [==============================] - 19s 135ms/step - loss: 1.3508e-05 - mean_absolute_error: 0.0025 - val_loss: 6.5235e-05 - val_mean_absolute_error: 0.0067
Epoch 24/30
140/140 [==============================] - 17s 119ms/step - loss: 1.5883e-05 - mean_absolute_error: 0.0026 - val_loss: 9.4824e-05 - val_mean_absolute_error: 0.0076
Epoch 25/30
140/140 [==============================] - 22s 154ms/step - loss: 1.8482e-05 - mean_absolute_error: 0.0029 - val_loss: 6.0889e-05 - val_mean_absolute_error: 0.0064
Epoch 26/30
140/140 [==============================] - 16s 115ms/step - loss: 1.5901e-05 - mean_absolute_error: 0.0027 - val_loss: 9.2104e-05 - val_mean_absolute_error: 0.0079
Epoch 27/30
140/140 [==============================] - 16s 115ms/step - loss: 1.6603e-05 - mean_absolute_error: 0.0027 - val_loss: 6.3611e-05 - val_mean_absolute_error: 0.0064
Epoch 28/30
140/140 [==============================] - 16s 115ms/step - loss: 1.2028e-05 - mean_absolute_error: 0.0024 - val_loss: 7.5925e-05 - val_mean_absolute_error: 0.0072
Epoch 29/30
140/140 [==============================] - 19s 134ms/step - loss: 1.4695e-05 - mean_absolute_error: 0.0026 - val_loss: 5.9952e-05 - val_mean_absolute_error: 0.0063
Epoch 30/30
140/140 [==============================] - 19s 135ms/step - loss: 1.1644e-05 - mean_absolute_error: 0.0022 - val_loss: 5.9832e-05 - val_mean_absolute_error: 0.0062

# Let's plot the mean_squared_error and mean_absolute_error using model history on the entire dataset
plot_loss_and_mae_validation(
    model_name="model_128_64_64_8_8_1_30_6_months_validation",
    model_history=history_128_64_64_8_8_1_30_6_months_validation,
)

Observations

Using six months of data as a window to predict the stock price, the training loss and training mean absolute error are decreasing gradually, whereas the validation loss and validation MAE are fluctuating, i.e., increasing and decreasing. The observed validation MAE is almost similar to that of compared to that of using 2 LSTM layers and 2 Dense layers with 6 months of data as a window. One option to potentially decrease the validation MAE further is to increase or decrease the model's complexity.

• 3 LSTM(128-64-64) 3 Dense(8-8-1) - 6 months
• Epochs: 30
• Batch Size: 16
• Training Loss: 1.1644e-05
• Validation Loss: 5.9832e-05
• Training MAE: 0.0022
• Validation MAE: 0.0062

Model Building with 4 LSTM and 3 Dense layers (8-16-16-32-8-8-1) and 30 epochs - 3 months

# Let's create a keras sequential model
model_8_16_16_32_8_8_1_30_3_months_validation = Sequential()

# Let's add two LSTM layer's to the model network
model_8_16_16_32_8_8_1_30_3_months_validation.add(
    LSTM(8, return_sequences=True, input_shape=(x_train_3_months.shape[1], 1))
)
model_8_16_16_32_8_8_1_30_3_months_validation.add(LSTM(16, return_sequences=True))
model_8_16_16_32_8_8_1_30_3_months_validation.add(LSTM(16, return_sequences=True))
model_8_16_16_32_8_8_1_30_3_months_validation.add(LSTM(32, return_sequences=False))

# Let's add dense layer to the model network
model_8_16_16_32_8_8_1_30_3_months_validation.add(Dense(8))
model_8_16_16_32_8_8_1_30_3_months_validation.add(Dense(8))
model_8_16_16_32_8_8_1_30_3_months_validation.add(Dense(1))

# Let's build the model using mean_squared_error as the loss function
# and mean_absolute_error as the evaluation metric during the compilation process
model_8_16_16_32_8_8_1_30_3_months_validation.compile(
    optimizer="adam", loss="mean_squared_error", metrics=["mean_absolute_error"]
)

# Let's define the callbacks for Model saving and Early stopping
cb_check = keras.callbacks.ModelCheckpoint(
    filepath="model_8_16_16_32_8_8_1_30_3_months_validation_checkpoint_filepath",
    save_best_only=True,
    monitor="val_loss",
)

cb_early = keras.callbacks.EarlyStopping(monitor="val_loss", patience=30)

# Let's fit the sequential model with input features and output label
history_8_16_16_32_8_8_1_30_3_months_validation = (
    model_8_16_16_32_8_8_1_30_3_months_validation.fit(
        x_train_3_months,
        y_train_3_months,
        batch_size=16,
        epochs=30,
        callbacks=[cb_check, cb_early],
        validation_data=(x_valid_3_months, y_valid_3_months),
    )
)

Epoch 1/30
146/146 [==============================] - 13s 70ms/step - loss: 7.2493e-04 - mean_absolute_error: 0.0151 - val_loss: 0.0011 - val_mean_absolute_error: 0.0272
Epoch 2/30
146/146 [==============================] - 5s 34ms/step - loss: 7.9525e-05 - mean_absolute_error: 0.0057 - val_loss: 0.0015 - val_mean_absolute_error: 0.0301
Epoch 3/30
146/146 [==============================] - 5s 34ms/step - loss: 6.7729e-05 - mean_absolute_error: 0.0053 - val_loss: 0.0016 - val_mean_absolute_error: 0.0315
Epoch 4/30
146/146 [==============================] - 8s 56ms/step - loss: 6.7200e-05 - mean_absolute_error: 0.0051 - val_loss: 9.1907e-04 - val_mean_absolute_error: 0.0241
Epoch 5/30
146/146 [==============================] - 8s 58ms/step - loss: 6.2734e-05 - mean_absolute_error: 0.0052 - val_loss: 7.1895e-04 - val_mean_absolute_error: 0.0217
Epoch 6/30
146/146 [==============================] - 5s 35ms/step - loss: 6.7805e-05 - mean_absolute_error: 0.0056 - val_loss: 0.0013 - val_mean_absolute_error: 0.0276
Epoch 7/30
146/146 [==============================] - 10s 70ms/step - loss: 5.6955e-05 - mean_absolute_error: 0.0048 - val_loss: 5.2291e-04 - val_mean_absolute_error: 0.0189
Epoch 8/30
146/146 [==============================] - 8s 56ms/step - loss: 5.3342e-05 - mean_absolute_error: 0.0047 - val_loss: 4.2211e-04 - val_mean_absolute_error: 0.0171
Epoch 9/30
146/146 [==============================] - 8s 57ms/step - loss: 4.6646e-05 - mean_absolute_error: 0.0046 - val_loss: 3.9730e-04 - val_mean_absolute_error: 0.0164
Epoch 10/30
146/146 [==============================] - 5s 34ms/step - loss: 4.9722e-05 - mean_absolute_error: 0.0046 - val_loss: 4.1199e-04 - val_mean_absolute_error: 0.0165
Epoch 11/30
146/146 [==============================] - 8s 58ms/step - loss: 3.9038e-05 - mean_absolute_error: 0.0041 - val_loss: 2.7621e-04 - val_mean_absolute_error: 0.0138
Epoch 12/30
146/146 [==============================] - 10s 69ms/step - loss: 3.3763e-05 - mean_absolute_error: 0.0039 - val_loss: 2.0293e-04 - val_mean_absolute_error: 0.0116
Epoch 13/30
146/146 [==============================] - 5s 34ms/step - loss: 3.6393e-05 - mean_absolute_error: 0.0042 - val_loss: 2.8367e-04 - val_mean_absolute_error: 0.0130
Epoch 14/30
146/146 [==============================] - 5s 34ms/step - loss: 3.7796e-05 - mean_absolute_error: 0.0041 - val_loss: 2.6072e-04 - val_mean_absolute_error: 0.0128
Epoch 15/30
146/146 [==============================] - 5s 33ms/step - loss: 2.9603e-05 - mean_absolute_error: 0.0037 - val_loss: 2.0549e-04 - val_mean_absolute_error: 0.0116
Epoch 16/30
146/146 [==============================] - 5s 33ms/step - loss: 3.2683e-05 - mean_absolute_error: 0.0038 - val_loss: 5.3862e-04 - val_mean_absolute_error: 0.0192
Epoch 17/30
146/146 [==============================] - 5s 33ms/step - loss: 3.9035e-05 - mean_absolute_error: 0.0046 - val_loss: 2.0621e-04 - val_mean_absolute_error: 0.0113
Epoch 18/30
146/146 [==============================] - 5s 33ms/step - loss: 3.7602e-05 - mean_absolute_error: 0.0043 - val_loss: 2.8093e-04 - val_mean_absolute_error: 0.0128
Epoch 19/30
146/146 [==============================] - 8s 58ms/step - loss: 2.6641e-05 - mean_absolute_error: 0.0035 - val_loss: 1.8380e-04 - val_mean_absolute_error: 0.0110
Epoch 20/30
146/146 [==============================] - 5s 34ms/step - loss: 2.6978e-05 - mean_absolute_error: 0.0035 - val_loss: 2.1307e-04 - val_mean_absolute_error: 0.0112
Epoch 21/30
146/146 [==============================] - 5s 33ms/step - loss: 2.7309e-05 - mean_absolute_error: 0.0035 - val_loss: 2.6020e-04 - val_mean_absolute_error: 0.0124
Epoch 22/30
146/146 [==============================] - 5s 33ms/step - loss: 3.1823e-05 - mean_absolute_error: 0.0038 - val_loss: 1.9350e-04 - val_mean_absolute_error: 0.0108
Epoch 23/30
146/146 [==============================] - 9s 59ms/step - loss: 3.3172e-05 - mean_absolute_error: 0.0041 - val_loss: 1.7473e-04 - val_mean_absolute_error: 0.0107
Epoch 24/30
146/146 [==============================] - 5s 33ms/step - loss: 2.9534e-05 - mean_absolute_error: 0.0038 - val_loss: 2.1252e-04 - val_mean_absolute_error: 0.0115
Epoch 25/30
146/146 [==============================] - 5s 34ms/step - loss: 3.1507e-05 - mean_absolute_error: 0.0037 - val_loss: 6.1884e-04 - val_mean_absolute_error: 0.0188
Epoch 26/30
146/146 [==============================] - 10s 65ms/step - loss: 3.5801e-05 - mean_absolute_error: 0.0041 - val_loss: 1.6523e-04 - val_mean_absolute_error: 0.0107
Epoch 27/30
146/146 [==============================] - 8s 56ms/step - loss: 2.2899e-05 - mean_absolute_error: 0.0032 - val_loss: 1.6198e-04 - val_mean_absolute_error: 0.0102
Epoch 28/30
146/146 [==============================] - 5s 35ms/step - loss: 2.2400e-05 - mean_absolute_error: 0.0032 - val_loss: 1.6976e-04 - val_mean_absolute_error: 0.0104
Epoch 29/30
146/146 [==============================] - 5s 34ms/step - loss: 1.9640e-05 - mean_absolute_error: 0.0029 - val_loss: 5.0514e-04 - val_mean_absolute_error: 0.0176
Epoch 30/30
146/146 [==============================] - 5s 34ms/step - loss: 1.8209e-05 - mean_absolute_error: 0.0028 - val_loss: 2.2960e-04 - val_mean_absolute_error: 0.0118

# Let's plot the mean_squared_error and mean_absolute_error using model history on the entire dataset
plot_loss_and_mae_validation(
    model_name="model_8_16_16_32_8_8_1_30_3_months_validation",
    model_history=history_8_16_16_32_8_8_1_30_3_months_validation,
)

Observations

Training loss and mean absolute error are decreasing gradually. Increasing the number of layers from three LSTM and three dense layer to four LSTM and three dense layers has not resulted in a decrease in mean absolute error when compared to one LSTM and one dense layer, which was trained for 50 epochs. To potentially further reduce the mean absolute error, increasing the number of epochs may result in a decrease in MAE.

• 3 LSTM(8-16-16-32) 3 Dense(8-8-1)
• Epochs: 50
• Batch Size: 1
• Training Loss: 1.2574e-04
• Mean Absolute Error: 0.0071

Model Building with 4 LSTM and 3 Dense layers (8-16-16-32-8-8-1) and 30 epochs - 6 months

# Let's create a keras sequential model
model_8_16_16_32_8_8_1_30_6_months_validation = Sequential()

# Let's add two LSTM layer's to the model network
model_8_16_16_32_8_8_1_30_6_months_validation.add(
    LSTM(8, return_sequences=True, input_shape=(x_train_6_months.shape[1], 1))
)
model_8_16_16_32_8_8_1_30_6_months_validation.add(LSTM(16, return_sequences=True))
model_8_16_16_32_8_8_1_30_6_months_validation.add(LSTM(16, return_sequences=True))
model_8_16_16_32_8_8_1_30_6_months_validation.add(LSTM(32, return_sequences=False))

# Let's add dense layer to the model network
model_8_16_16_32_8_8_1_30_6_months_validation.add(Dense(8))
model_8_16_16_32_8_8_1_30_6_months_validation.add(Dense(8))
model_8_16_16_32_8_8_1_30_6_months_validation.add(Dense(1))

# Let's build the model using mean_squared_error as the loss function
# and mean_absolute_error as the evaluation metric during the compilation process
model_8_16_16_32_8_8_1_30_6_months_validation.compile(
    optimizer="adam", loss="mean_squared_error", metrics=["mean_absolute_error"]
)

# Let's define the callbacks for Model saving and Early stopping
cb_check = keras.callbacks.ModelCheckpoint(
    filepath="model_8_16_16_32_8_8_1_30_6_months_validation_checkpoint_filepath",
    save_best_only=True,
    monitor="val_loss",
)

cb_early = keras.callbacks.EarlyStopping(monitor="val_loss", patience=30)

# Let's fit the sequential model with input features and output label
history_8_16_16_32_8_8_1_30_6_months_validation = (
    model_8_16_16_32_8_8_1_30_6_months_validation.fit(
        x_train_6_months,
        y_train_6_months,
        batch_size=16,
        epochs=30,
        callbacks=[cb_check, cb_early],
        validation_data=(x_valid_6_months, y_valid_6_months),
    )
)

Epoch 1/30
140/140 [==============================] - 17s 102ms/step - loss: 5.2403e-04 - mean_absolute_error: 0.0125 - val_loss: 0.0012 - val_mean_absolute_error: 0.0285
Epoch 2/30
140/140 [==============================] - 13s 92ms/step - loss: 9.5819e-05 - mean_absolute_error: 0.0061 - val_loss: 0.0011 - val_mean_absolute_error: 0.0268
Epoch 3/30
140/140 [==============================] - 13s 90ms/step - loss: 9.4477e-05 - mean_absolute_error: 0.0065 - val_loss: 9.2375e-04 - val_mean_absolute_error: 0.0259
Epoch 4/30
140/140 [==============================] - 15s 104ms/step - loss: 1.0992e-04 - mean_absolute_error: 0.0070 - val_loss: 8.4143e-04 - val_mean_absolute_error: 0.0243
Epoch 5/30
140/140 [==============================] - 13s 90ms/step - loss: 7.9623e-05 - mean_absolute_error: 0.0058 - val_loss: 7.3670e-04 - val_mean_absolute_error: 0.0230
Epoch 6/30
140/140 [==============================] - 9s 67ms/step - loss: 7.8633e-05 - mean_absolute_error: 0.0056 - val_loss: 8.3744e-04 - val_mean_absolute_error: 0.0235
Epoch 7/30
140/140 [==============================] - 13s 91ms/step - loss: 8.1935e-05 - mean_absolute_error: 0.0060 - val_loss: 7.3503e-04 - val_mean_absolute_error: 0.0222
Epoch 8/30
140/140 [==============================] - 14s 98ms/step - loss: 6.9296e-05 - mean_absolute_error: 0.0053 - val_loss: 6.5486e-04 - val_mean_absolute_error: 0.0213
Epoch 9/30
140/140 [==============================] - 10s 70ms/step - loss: 7.6519e-05 - mean_absolute_error: 0.0056 - val_loss: 8.6934e-04 - val_mean_absolute_error: 0.0230
Epoch 10/30
140/140 [==============================] - 15s 109ms/step - loss: 6.9728e-05 - mean_absolute_error: 0.0054 - val_loss: 5.8623e-04 - val_mean_absolute_error: 0.0196
Epoch 11/30
140/140 [==============================] - 13s 93ms/step - loss: 5.4132e-05 - mean_absolute_error: 0.0048 - val_loss: 4.6029e-04 - val_mean_absolute_error: 0.0181
Epoch 12/30
140/140 [==============================] - 13s 92ms/step - loss: 5.7346e-05 - mean_absolute_error: 0.0050 - val_loss: 4.2058e-04 - val_mean_absolute_error: 0.0170
Epoch 13/30
140/140 [==============================] - 14s 102ms/step - loss: 4.1986e-05 - mean_absolute_error: 0.0041 - val_loss: 3.2710e-04 - val_mean_absolute_error: 0.0154
Epoch 14/30
140/140 [==============================] - 13s 90ms/step - loss: 4.2947e-05 - mean_absolute_error: 0.0044 - val_loss: 2.6610e-04 - val_mean_absolute_error: 0.0134
Epoch 15/30
140/140 [==============================] - 13s 92ms/step - loss: 3.1485e-05 - mean_absolute_error: 0.0038 - val_loss: 2.3336e-04 - val_mean_absolute_error: 0.0130
Epoch 16/30
140/140 [==============================] - 9s 67ms/step - loss: 2.7288e-05 - mean_absolute_error: 0.0033 - val_loss: 2.7260e-04 - val_mean_absolute_error: 0.0137
Epoch 17/30
140/140 [==============================] - 9s 67ms/step - loss: 3.7175e-05 - mean_absolute_error: 0.0042 - val_loss: 2.8429e-04 - val_mean_absolute_error: 0.0137
Epoch 18/30
140/140 [==============================] - 10s 69ms/step - loss: 4.1787e-05 - mean_absolute_error: 0.0045 - val_loss: 2.4806e-04 - val_mean_absolute_error: 0.0139
Epoch 19/30
140/140 [==============================] - 15s 104ms/step - loss: 2.8241e-05 - mean_absolute_error: 0.0035 - val_loss: 2.0975e-04 - val_mean_absolute_error: 0.0124
Epoch 20/30
140/140 [==============================] - 9s 66ms/step - loss: 3.3354e-05 - mean_absolute_error: 0.0038 - val_loss: 4.5851e-04 - val_mean_absolute_error: 0.0186
Epoch 21/30
140/140 [==============================] - 13s 90ms/step - loss: 4.1337e-05 - mean_absolute_error: 0.0043 - val_loss: 2.0919e-04 - val_mean_absolute_error: 0.0121
Epoch 22/30
140/140 [==============================] - 9s 66ms/step - loss: 3.6391e-05 - mean_absolute_error: 0.0042 - val_loss: 2.3651e-04 - val_mean_absolute_error: 0.0124
Epoch 23/30
140/140 [==============================] - 13s 90ms/step - loss: 3.1137e-05 - mean_absolute_error: 0.0037 - val_loss: 1.8176e-04 - val_mean_absolute_error: 0.0114
Epoch 24/30
140/140 [==============================] - 9s 67ms/step - loss: 2.9830e-05 - mean_absolute_error: 0.0035 - val_loss: 1.9414e-04 - val_mean_absolute_error: 0.0115
Epoch 25/30
140/140 [==============================] - 9s 66ms/step - loss: 3.0443e-05 - mean_absolute_error: 0.0038 - val_loss: 2.2664e-04 - val_mean_absolute_error: 0.0130
Epoch 26/30
140/140 [==============================] - 19s 136ms/step - loss: 3.1566e-05 - mean_absolute_error: 0.0037 - val_loss: 1.7541e-04 - val_mean_absolute_error: 0.0109
Epoch 27/30
140/140 [==============================] - 9s 67ms/step - loss: 2.7969e-05 - mean_absolute_error: 0.0035 - val_loss: 2.1094e-04 - val_mean_absolute_error: 0.0121
Epoch 28/30
140/140 [==============================] - 9s 67ms/step - loss: 2.6681e-05 - mean_absolute_error: 0.0035 - val_loss: 3.7428e-04 - val_mean_absolute_error: 0.0158
Epoch 29/30
140/140 [==============================] - 10s 68ms/step - loss: 2.7643e-05 - mean_absolute_error: 0.0035 - val_loss: 1.9925e-04 - val_mean_absolute_error: 0.0116
Epoch 30/30
140/140 [==============================] - 13s 94ms/step - loss: 2.5782e-05 - mean_absolute_error: 0.0033 - val_loss: 1.6052e-04 - val_mean_absolute_error: 0.0105

# Let's plot the mean_squared_error and mean_absolute_error using model history on the entire dataset
plot_loss_and_mae_validation(
    model_name="model_8_16_16_32_8_8_1_30_6_months_validation",
    model_history=history_8_16_16_32_8_8_1_30_6_months_validation,
)

Observations

Training loss and mean absolute error are decreasing gradually. Increasing the number of layers from three LSTM and three dense layer to four LSTM and three dense layers has not resulted in a decrease in mean absolute error when compared to one LSTM and one dense layer, which was trained for 50 epochs. To potentially further reduce the mean absolute error, increasing the number of epochs may result in a decrease in MAE.

• 3 LSTM(8-16-16-32) 3 Dense(8-8-1)
• Epochs: 50
• Batch Size: 1
• Training Loss: 1.2574e-04
• Mean Absolute Error: 0.0071

Model Building with 4 LSTM and 4 Dense layers (8-16-16-32-32-16-16-1) and 30 epochs - 3 months

# Let's create a keras sequential model
model_8_16_16_32_32_16_16_1_30_3_months_validation = Sequential()

# Let's add two LSTM layer's to the model network
model_8_16_16_32_32_16_16_1_30_3_months_validation.add(
    LSTM(8, return_sequences=True, input_shape=(x_train_3_months.shape[1], 1))
)
model_8_16_16_32_32_16_16_1_30_3_months_validation.add(LSTM(16, return_sequences=True))
model_8_16_16_32_32_16_16_1_30_3_months_validation.add(LSTM(16, return_sequences=True))
model_8_16_16_32_32_16_16_1_30_3_months_validation.add(LSTM(32, return_sequences=False))

# Let's add dense layer to the model network
model_8_16_16_32_32_16_16_1_30_3_months_validation.add(Dense(32))
model_8_16_16_32_32_16_16_1_30_3_months_validation.add(Dense(16))
model_8_16_16_32_32_16_16_1_30_3_months_validation.add(Dense(16))
model_8_16_16_32_32_16_16_1_30_3_months_validation.add(Dense(1))

# Let's build the model using mean_squared_error as the loss function
# and mean_absolute_error as the evaluation metric during the compilation process
model_8_16_16_32_32_16_16_1_30_3_months_validation.compile(
    optimizer="adam", loss="mean_squared_error", metrics=["mean_absolute_error"]
)

# Let's define the callbacks for Model saving and Early stopping
cb_check = keras.callbacks.ModelCheckpoint(
    filepath="model_8_16_16_32_32_16_16_1_30_3_months_validation_checkpoint_filepath",
    save_best_only=True,
    monitor="val_loss",
)

cb_early = keras.callbacks.EarlyStopping(monitor="val_loss", patience=30)

# Let's fit the sequential model with input features and output label
history_8_16_16_32_32_16_16_1_30_3_months_validation = (
    model_8_16_16_32_32_16_16_1_30_3_months_validation.fit(
        x_train_3_months,
        y_train_3_months,
        batch_size=16,
        epochs=30,
        callbacks=[cb_check, cb_early],
        validation_data=(x_valid_3_months, y_valid_3_months),
    )
)

Epoch 1/30
146/146 [==============================] - 15s 74ms/step - loss: 3.5111e-04 - mean_absolute_error: 0.0102 - val_loss: 0.0010 - val_mean_absolute_error: 0.0264
Epoch 2/30
146/146 [==============================] - 14s 99ms/step - loss: 9.1027e-05 - mean_absolute_error: 0.0065 - val_loss: 8.1941e-04 - val_mean_absolute_error: 0.0240
Epoch 3/30
146/146 [==============================] - 5s 34ms/step - loss: 9.7719e-05 - mean_absolute_error: 0.0066 - val_loss: 0.0046 - val_mean_absolute_error: 0.0572
Epoch 4/30
146/146 [==============================] - 5s 34ms/step - loss: 1.0607e-04 - mean_absolute_error: 0.0067 - val_loss: 9.3181e-04 - val_mean_absolute_error: 0.0246
Epoch 5/30
146/146 [==============================] - 9s 61ms/step - loss: 7.3099e-05 - mean_absolute_error: 0.0057 - val_loss: 6.6123e-04 - val_mean_absolute_error: 0.0218
Epoch 6/30
146/146 [==============================] - 5s 35ms/step - loss: 6.6972e-05 - mean_absolute_error: 0.0053 - val_loss: 7.1324e-04 - val_mean_absolute_error: 0.0220
Epoch 7/30
146/146 [==============================] - 5s 35ms/step - loss: 7.9622e-05 - mean_absolute_error: 0.0062 - val_loss: 0.0010 - val_mean_absolute_error: 0.0252
Epoch 8/30
146/146 [==============================] - 5s 35ms/step - loss: 6.4335e-05 - mean_absolute_error: 0.0053 - val_loss: 8.9891e-04 - val_mean_absolute_error: 0.0235
Epoch 9/30
146/146 [==============================] - 5s 34ms/step - loss: 6.5121e-05 - mean_absolute_error: 0.0054 - val_loss: 8.8368e-04 - val_mean_absolute_error: 0.0234
Epoch 10/30
146/146 [==============================] - 5s 35ms/step - loss: 9.3640e-05 - mean_absolute_error: 0.0066 - val_loss: 7.4134e-04 - val_mean_absolute_error: 0.0216
Epoch 11/30
146/146 [==============================] - 9s 60ms/step - loss: 5.2029e-05 - mean_absolute_error: 0.0047 - val_loss: 4.5651e-04 - val_mean_absolute_error: 0.0176
Epoch 12/30
146/146 [==============================] - 10s 69ms/step - loss: 4.6093e-05 - mean_absolute_error: 0.0045 - val_loss: 3.3373e-04 - val_mean_absolute_error: 0.0154
Epoch 13/30
146/146 [==============================] - 5s 34ms/step - loss: 3.4056e-05 - mean_absolute_error: 0.0040 - val_loss: 6.9304e-04 - val_mean_absolute_error: 0.0214
Epoch 14/30
146/146 [==============================] - 5s 35ms/step - loss: 3.1748e-05 - mean_absolute_error: 0.0038 - val_loss: 4.2372e-04 - val_mean_absolute_error: 0.0162
Epoch 15/30
146/146 [==============================] - 9s 62ms/step - loss: 3.6422e-05 - mean_absolute_error: 0.0043 - val_loss: 2.5936e-04 - val_mean_absolute_error: 0.0127
Epoch 16/30
146/146 [==============================] - 5s 35ms/step - loss: 3.2129e-05 - mean_absolute_error: 0.0038 - val_loss: 4.4474e-04 - val_mean_absolute_error: 0.0167
Epoch 17/30
146/146 [==============================] - 5s 34ms/step - loss: 4.7879e-05 - mean_absolute_error: 0.0047 - val_loss: 2.6073e-04 - val_mean_absolute_error: 0.0127
Epoch 18/30
146/146 [==============================] - 5s 35ms/step - loss: 3.0191e-05 - mean_absolute_error: 0.0037 - val_loss: 3.5963e-04 - val_mean_absolute_error: 0.0145
Epoch 19/30
146/146 [==============================] - 5s 36ms/step - loss: 3.3093e-05 - mean_absolute_error: 0.0039 - val_loss: 5.8113e-04 - val_mean_absolute_error: 0.0190
Epoch 20/30
146/146 [==============================] - 5s 37ms/step - loss: 2.8153e-05 - mean_absolute_error: 0.0036 - val_loss: 8.0967e-04 - val_mean_absolute_error: 0.0234
Epoch 21/30
146/146 [==============================] - 5s 35ms/step - loss: 3.1698e-05 - mean_absolute_error: 0.0039 - val_loss: 3.2963e-04 - val_mean_absolute_error: 0.0140
Epoch 22/30
146/146 [==============================] - 9s 63ms/step - loss: 2.6251e-05 - mean_absolute_error: 0.0033 - val_loss: 2.5549e-04 - val_mean_absolute_error: 0.0127
Epoch 23/30
146/146 [==============================] - 9s 63ms/step - loss: 2.8255e-05 - mean_absolute_error: 0.0036 - val_loss: 2.4474e-04 - val_mean_absolute_error: 0.0119
Epoch 24/30
146/146 [==============================] - 13s 86ms/step - loss: 2.6515e-05 - mean_absolute_error: 0.0034 - val_loss: 2.1504e-04 - val_mean_absolute_error: 0.0115
Epoch 25/30
146/146 [==============================] - 5s 34ms/step - loss: 2.4870e-05 - mean_absolute_error: 0.0034 - val_loss: 2.2451e-04 - val_mean_absolute_error: 0.0120
Epoch 26/30
146/146 [==============================] - 5s 34ms/step - loss: 2.5451e-05 - mean_absolute_error: 0.0035 - val_loss: 3.3164e-04 - val_mean_absolute_error: 0.0143
Epoch 27/30
146/146 [==============================] - 9s 60ms/step - loss: 4.3300e-05 - mean_absolute_error: 0.0048 - val_loss: 1.9682e-04 - val_mean_absolute_error: 0.0115
Epoch 28/30
146/146 [==============================] - 9s 60ms/step - loss: 3.5540e-05 - mean_absolute_error: 0.0042 - val_loss: 1.9011e-04 - val_mean_absolute_error: 0.0110
Epoch 29/30
146/146 [==============================] - 10s 68ms/step - loss: 2.8973e-05 - mean_absolute_error: 0.0038 - val_loss: 1.5813e-04 - val_mean_absolute_error: 0.0101
Epoch 30/30
146/146 [==============================] - 5s 34ms/step - loss: 2.6600e-05 - mean_absolute_error: 0.0036 - val_loss: 3.5716e-04 - val_mean_absolute_error: 0.0149

# Let's plot the mean_squared_error and mean_absolute_error using model history on the entire dataset
plot_loss_and_mae_validation(
    model_name="model_8_16_16_32_32_16_16_1_30_3_months_validation",
    model_history=history_8_16_16_32_32_16_16_1_30_3_months_validation,
)

Observations

Training loss and mean absolute error are decreasing gradually. Increasing the number of Dense layers and neurons has not resulted in a decrease in mean absolute error when compared to one LSTM and one dense layer, which was trained for 50 epochs. To potentially further reduce the mean absolute error, increasing the number of LSTM layers and neurons may result in a decrease in MAE.

• 4 LSTM(8-16-16-32) 4 Dense(32-32-16-16-1)
• Epochs: 50
• Batch Size: 1
• Training Loss: 1.1192e-04
• Mean Absolute Error: 0.0070

Model Building with 4 LSTM and 4 Dense layers (8-16-16-32-32-16-16-1) and 30 epochs - 6 months

# Let's create a keras sequential model
model_8_16_16_32_32_16_16_1_30_6_months_validation = Sequential()

# Let's add two LSTM layer's to the model network
model_8_16_16_32_32_16_16_1_30_6_months_validation.add(
    LSTM(8, return_sequences=True, input_shape=(x_train_6_months.shape[1], 1))
)
model_8_16_16_32_32_16_16_1_30_6_months_validation.add(LSTM(16, return_sequences=True))
model_8_16_16_32_32_16_16_1_30_6_months_validation.add(LSTM(16, return_sequences=True))
model_8_16_16_32_32_16_16_1_30_6_months_validation.add(LSTM(32, return_sequences=False))

# Let's add dense layer to the model network
model_8_16_16_32_32_16_16_1_30_6_months_validation.add(Dense(32))
model_8_16_16_32_32_16_16_1_30_6_months_validation.add(Dense(16))
model_8_16_16_32_32_16_16_1_30_6_months_validation.add(Dense(16))
model_8_16_16_32_32_16_16_1_30_6_months_validation.add(Dense(1))

# Let's build the model using mean_squared_error as the loss function
# and mean_absolute_error as the evaluation metric during the compilation process
model_8_16_16_32_32_16_16_1_30_6_months_validation.compile(
    optimizer="adam", loss="mean_squared_error", metrics=["mean_absolute_error"]
)

# Let's define the callbacks for Model saving and Early stopping
cb_check = keras.callbacks.ModelCheckpoint(
    filepath="model_8_16_16_32_32_16_16_1_30_6_months_validation_checkpoint_filepath",
    save_best_only=True,
    monitor="val_loss",
)

cb_early = keras.callbacks.EarlyStopping(monitor="val_loss", patience=30)

# Let's fit the sequential model with input features and output label
history_8_16_16_32_32_16_16_1_30_6_months_validation = (
    model_8_16_16_32_32_16_16_1_30_6_months_validation.fit(
        x_train_6_months,
        y_train_6_months,
        batch_size=16,
        epochs=30,
        callbacks=[cb_check, cb_early],
        validation_data=(x_valid_6_months, y_valid_6_months),
    )
)

Epoch 1/30
140/140 [==============================] - 16s 97ms/step - loss: 2.7461e-04 - mean_absolute_error: 0.0097 - val_loss: 0.0016 - val_mean_absolute_error: 0.0314
Epoch 2/30
140/140 [==============================] - 13s 93ms/step - loss: 1.1742e-04 - mean_absolute_error: 0.0073 - val_loss: 7.7340e-04 - val_mean_absolute_error: 0.0243
Epoch 3/30
140/140 [==============================] - 15s 105ms/step - loss: 8.2311e-05 - mean_absolute_error: 0.0062 - val_loss: 7.6410e-04 - val_mean_absolute_error: 0.0236
Epoch 4/30
140/140 [==============================] - 13s 91ms/step - loss: 8.9169e-05 - mean_absolute_error: 0.0064 - val_loss: 6.8764e-04 - val_mean_absolute_error: 0.0226
Epoch 5/30
140/140 [==============================] - 10s 70ms/step - loss: 8.9308e-05 - mean_absolute_error: 0.0064 - val_loss: 7.2488e-04 - val_mean_absolute_error: 0.0232
Epoch 6/30
140/140 [==============================] - 10s 70ms/step - loss: 7.3532e-05 - mean_absolute_error: 0.0055 - val_loss: 8.3338e-04 - val_mean_absolute_error: 0.0226
Epoch 7/30
140/140 [==============================] - 10s 69ms/step - loss: 6.8717e-05 - mean_absolute_error: 0.0054 - val_loss: 7.1793e-04 - val_mean_absolute_error: 0.0211
Epoch 8/30
140/140 [==============================] - 14s 98ms/step - loss: 7.8559e-05 - mean_absolute_error: 0.0061 - val_loss: 5.4361e-04 - val_mean_absolute_error: 0.0200
Epoch 9/30
140/140 [==============================] - 10s 70ms/step - loss: 7.2025e-05 - mean_absolute_error: 0.0058 - val_loss: 6.0588e-04 - val_mean_absolute_error: 0.0195
Epoch 10/30
140/140 [==============================] - 14s 97ms/step - loss: 6.9979e-05 - mean_absolute_error: 0.0056 - val_loss: 5.0553e-04 - val_mean_absolute_error: 0.0180
Epoch 11/30
140/140 [==============================] - 10s 71ms/step - loss: 9.4650e-05 - mean_absolute_error: 0.0069 - val_loss: 5.1107e-04 - val_mean_absolute_error: 0.0180
Epoch 12/30
140/140 [==============================] - 15s 108ms/step - loss: 5.8316e-05 - mean_absolute_error: 0.0051 - val_loss: 3.6879e-04 - val_mean_absolute_error: 0.0165
Epoch 13/30
140/140 [==============================] - 10s 68ms/step - loss: 4.3259e-05 - mean_absolute_error: 0.0045 - val_loss: 4.1849e-04 - val_mean_absolute_error: 0.0159
Epoch 14/30
140/140 [==============================] - 13s 97ms/step - loss: 3.8214e-05 - mean_absolute_error: 0.0040 - val_loss: 2.8898e-04 - val_mean_absolute_error: 0.0137
Epoch 15/30
140/140 [==============================] - 10s 70ms/step - loss: 4.0612e-05 - mean_absolute_error: 0.0044 - val_loss: 8.5581e-04 - val_mean_absolute_error: 0.0250
Epoch 16/30
140/140 [==============================] - 13s 94ms/step - loss: 3.5941e-05 - mean_absolute_error: 0.0039 - val_loss: 2.8071e-04 - val_mean_absolute_error: 0.0135
Epoch 17/30
140/140 [==============================] - 15s 109ms/step - loss: 2.7400e-05 - mean_absolute_error: 0.0034 - val_loss: 2.0990e-04 - val_mean_absolute_error: 0.0120
Epoch 18/30
140/140 [==============================] - 13s 94ms/step - loss: 4.2792e-05 - mean_absolute_error: 0.0045 - val_loss: 1.9832e-04 - val_mean_absolute_error: 0.0116
Epoch 19/30
140/140 [==============================] - 10s 70ms/step - loss: 4.3826e-05 - mean_absolute_error: 0.0044 - val_loss: 3.0607e-04 - val_mean_absolute_error: 0.0138
Epoch 20/30
140/140 [==============================] - 9s 67ms/step - loss: 4.1193e-05 - mean_absolute_error: 0.0047 - val_loss: 2.6037e-04 - val_mean_absolute_error: 0.0130
Epoch 21/30
140/140 [==============================] - 9s 66ms/step - loss: 3.4519e-05 - mean_absolute_error: 0.0041 - val_loss: 2.1304e-04 - val_mean_absolute_error: 0.0123
Epoch 22/30
140/140 [==============================] - 9s 66ms/step - loss: 4.0828e-05 - mean_absolute_error: 0.0045 - val_loss: 3.1915e-04 - val_mean_absolute_error: 0.0150
Epoch 23/30
140/140 [==============================] - 13s 91ms/step - loss: 2.9943e-05 - mean_absolute_error: 0.0036 - val_loss: 1.6721e-04 - val_mean_absolute_error: 0.0106
Epoch 24/30
140/140 [==============================] - 9s 67ms/step - loss: 3.4985e-05 - mean_absolute_error: 0.0040 - val_loss: 2.0194e-04 - val_mean_absolute_error: 0.0121
Epoch 25/30
140/140 [==============================] - 10s 68ms/step - loss: 2.9480e-05 - mean_absolute_error: 0.0038 - val_loss: 3.1182e-04 - val_mean_absolute_error: 0.0150
Epoch 26/30
140/140 [==============================] - 9s 68ms/step - loss: 3.7755e-05 - mean_absolute_error: 0.0044 - val_loss: 1.8858e-04 - val_mean_absolute_error: 0.0113
Epoch 27/30
140/140 [==============================] - 9s 67ms/step - loss: 2.7460e-05 - mean_absolute_error: 0.0036 - val_loss: 2.8415e-04 - val_mean_absolute_error: 0.0141
Epoch 28/30
140/140 [==============================] - 9s 66ms/step - loss: 3.1299e-05 - mean_absolute_error: 0.0038 - val_loss: 1.7370e-04 - val_mean_absolute_error: 0.0111
Epoch 29/30
140/140 [==============================] - 9s 66ms/step - loss: 3.2733e-05 - mean_absolute_error: 0.0039 - val_loss: 3.0168e-04 - val_mean_absolute_error: 0.0141
Epoch 30/30
140/140 [==============================] - 9s 66ms/step - loss: 2.8791e-05 - mean_absolute_error: 0.0037 - val_loss: 2.0967e-04 - val_mean_absolute_error: 0.0116

# Let's plot the mean_squared_error and mean_absolute_error using model history on the entire dataset
plot_loss_and_mae_validation(
    model_name="model_8_16_16_32_32_16_16_1_30_6_months_validation",
    model_history=history_8_16_16_32_32_16_16_1_30_6_months_validation,
)

Observations

Training loss and mean absolute error are decreasing gradually. Increasing the number of Dense layers and neurons has not resulted in a decrease in mean absolute error when compared to one LSTM and one dense layer, which was trained for 50 epochs. To potentially further reduce the mean absolute error, increasing the number of LSTM layers and neurons may result in a decrease in MAE.

• 4 LSTM(8-16-16-32) 4 Dense(32-32-16-16-1)
• Epochs: 50
• Batch Size: 1
• Training Loss: 1.1192e-04
• Mean Absolute Error: 0.0070

Model Building with 5 LSTM and 3 Dense layers (8-16-16-32-64-8-8-1) and 30 epochs - 3 months

# Let's create a keras sequential model
model_8_16_16_32_64_8_8_1_30_3_months_validation = Sequential()

# Let's add two LSTM layer's to the model network
model_8_16_16_32_64_8_8_1_30_3_months_validation.add(
    LSTM(8, return_sequences=True, input_shape=(x_train_3_months.shape[1], 1))
)
model_8_16_16_32_64_8_8_1_30_3_months_validation.add(LSTM(16, return_sequences=True))
model_8_16_16_32_64_8_8_1_30_3_months_validation.add(LSTM(16, return_sequences=True))
model_8_16_16_32_64_8_8_1_30_3_months_validation.add(LSTM(32, return_sequences=True))
model_8_16_16_32_64_8_8_1_30_3_months_validation.add(LSTM(64, return_sequences=False))

# Let's add dense layer to the model network
model_8_16_16_32_64_8_8_1_30_3_months_validation.add(Dense(8))
model_8_16_16_32_64_8_8_1_30_3_months_validation.add(Dense(8))
model_8_16_16_32_64_8_8_1_30_3_months_validation.add(Dense(1))

# Let's build the model using mean_squared_error as the loss function
# and mean_absolute_error as the evaluation metric during the compilation process
model_8_16_16_32_64_8_8_1_30_3_months_validation.compile(
    optimizer="adam", loss="mean_squared_error", metrics=["mean_absolute_error"]
)

# Let's define the callbacks for Model saving and Early stopping
cb_check = keras.callbacks.ModelCheckpoint(
    filepath="model_8_16_16_32_64_8_8_1_30_3_months_validation_checkpoint_filepath",
    save_best_only=True,
    monitor="val_loss",
)

cb_early = keras.callbacks.EarlyStopping(monitor="val_loss", patience=30)

# Let's fit the sequential model with input features and output label
history_8_16_16_32_64_8_8_1_30_3_months_validation = (
    model_8_16_16_32_64_8_8_1_30_3_months_validation.fit(
        x_train_3_months,
        y_train_3_months,
        batch_size=16,
        epochs=30,
        callbacks=[cb_check, cb_early],
        validation_data=(x_valid_3_months, y_valid_3_months),
    )
)

Epoch 1/30
146/146 [==============================] - 17s 98ms/step - loss: 3.6395e-04 - mean_absolute_error: 0.0118 - val_loss: 0.0014 - val_mean_absolute_error: 0.0312
Epoch 2/30
146/146 [==============================] - 7s 51ms/step - loss: 1.1566e-04 - mean_absolute_error: 0.0071 - val_loss: 0.0019 - val_mean_absolute_error: 0.0344
Epoch 3/30
146/146 [==============================] - 11s 77ms/step - loss: 9.8903e-05 - mean_absolute_error: 0.0066 - val_loss: 0.0011 - val_mean_absolute_error: 0.0284
Epoch 4/30
146/146 [==============================] - 7s 48ms/step - loss: 9.8803e-05 - mean_absolute_error: 0.0064 - val_loss: 0.0015 - val_mean_absolute_error: 0.0311
Epoch 5/30
146/146 [==============================] - 7s 50ms/step - loss: 1.0905e-04 - mean_absolute_error: 0.0071 - val_loss: 0.0018 - val_mean_absolute_error: 0.0338
Epoch 6/30
146/146 [==============================] - 13s 92ms/step - loss: 7.5534e-05 - mean_absolute_error: 0.0056 - val_loss: 7.7510e-04 - val_mean_absolute_error: 0.0235
Epoch 7/30
146/146 [==============================] - 7s 49ms/step - loss: 6.9688e-05 - mean_absolute_error: 0.0055 - val_loss: 9.5502e-04 - val_mean_absolute_error: 0.0248
Epoch 8/30
146/146 [==============================] - 12s 83ms/step - loss: 8.5421e-05 - mean_absolute_error: 0.0062 - val_loss: 6.2736e-04 - val_mean_absolute_error: 0.0212
Epoch 9/30
146/146 [==============================] - 12s 83ms/step - loss: 5.6732e-05 - mean_absolute_error: 0.0049 - val_loss: 5.3703e-04 - val_mean_absolute_error: 0.0191
Epoch 10/30
146/146 [==============================] - 8s 51ms/step - loss: 5.7086e-05 - mean_absolute_error: 0.0049 - val_loss: 8.1607e-04 - val_mean_absolute_error: 0.0227
Epoch 11/30
146/146 [==============================] - 7s 50ms/step - loss: 4.9616e-05 - mean_absolute_error: 0.0047 - val_loss: 6.8684e-04 - val_mean_absolute_error: 0.0210
Epoch 12/30
146/146 [==============================] - 12s 79ms/step - loss: 3.8057e-05 - mean_absolute_error: 0.0040 - val_loss: 4.9357e-04 - val_mean_absolute_error: 0.0174
Epoch 13/30
146/146 [==============================] - 7s 50ms/step - loss: 4.2912e-05 - mean_absolute_error: 0.0044 - val_loss: 4.9703e-04 - val_mean_absolute_error: 0.0165
Epoch 14/30
146/146 [==============================] - 14s 95ms/step - loss: 3.5305e-05 - mean_absolute_error: 0.0040 - val_loss: 3.5769e-04 - val_mean_absolute_error: 0.0156
Epoch 15/30
146/146 [==============================] - 7s 50ms/step - loss: 4.1426e-05 - mean_absolute_error: 0.0042 - val_loss: 7.8867e-04 - val_mean_absolute_error: 0.0215
Epoch 16/30
146/146 [==============================] - 12s 84ms/step - loss: 3.2924e-05 - mean_absolute_error: 0.0037 - val_loss: 3.4762e-04 - val_mean_absolute_error: 0.0146
Epoch 17/30
146/146 [==============================] - 7s 48ms/step - loss: 2.9540e-05 - mean_absolute_error: 0.0036 - val_loss: 4.7968e-04 - val_mean_absolute_error: 0.0167
Epoch 18/30
146/146 [==============================] - 7s 49ms/step - loss: 3.6460e-05 - mean_absolute_error: 0.0042 - val_loss: 6.6975e-04 - val_mean_absolute_error: 0.0199
Epoch 19/30
146/146 [==============================] - 15s 104ms/step - loss: 3.5014e-05 - mean_absolute_error: 0.0040 - val_loss: 2.3597e-04 - val_mean_absolute_error: 0.0125
Epoch 20/30
146/146 [==============================] - 8s 51ms/step - loss: 6.3325e-05 - mean_absolute_error: 0.0055 - val_loss: 3.9221e-04 - val_mean_absolute_error: 0.0154
Epoch 21/30
146/146 [==============================] - 7s 50ms/step - loss: 4.4711e-05 - mean_absolute_error: 0.0043 - val_loss: 3.6130e-04 - val_mean_absolute_error: 0.0149
Epoch 22/30
146/146 [==============================] - 7s 50ms/step - loss: 4.3229e-05 - mean_absolute_error: 0.0047 - val_loss: 2.9560e-04 - val_mean_absolute_error: 0.0146
Epoch 23/30
146/146 [==============================] - 7s 51ms/step - loss: 3.4653e-05 - mean_absolute_error: 0.0040 - val_loss: 2.4870e-04 - val_mean_absolute_error: 0.0129
Epoch 24/30
146/146 [==============================] - 7s 49ms/step - loss: 2.7861e-05 - mean_absolute_error: 0.0035 - val_loss: 3.5828e-04 - val_mean_absolute_error: 0.0147
Epoch 25/30
146/146 [==============================] - 7s 49ms/step - loss: 2.8259e-05 - mean_absolute_error: 0.0034 - val_loss: 2.5763e-04 - val_mean_absolute_error: 0.0123
Epoch 26/30
146/146 [==============================] - 7s 51ms/step - loss: 2.9149e-05 - mean_absolute_error: 0.0036 - val_loss: 2.7450e-04 - val_mean_absolute_error: 0.0127
Epoch 27/30
146/146 [==============================] - 12s 83ms/step - loss: 2.9177e-05 - mean_absolute_error: 0.0035 - val_loss: 2.2621e-04 - val_mean_absolute_error: 0.0117
Epoch 28/30
146/146 [==============================] - 8s 52ms/step - loss: 2.6328e-05 - mean_absolute_error: 0.0032 - val_loss: 4.7275e-04 - val_mean_absolute_error: 0.0171
Epoch 29/30
146/146 [==============================] - 7s 50ms/step - loss: 3.4058e-05 - mean_absolute_error: 0.0039 - val_loss: 2.2705e-04 - val_mean_absolute_error: 0.0117
Epoch 30/30
146/146 [==============================] - 7s 49ms/step - loss: 3.7786e-05 - mean_absolute_error: 0.0044 - val_loss: 3.5857e-04 - val_mean_absolute_error: 0.0149

# Let's plot the mean_squared_error and mean_absolute_error using model history on the entire dataset
plot_loss_and_mae_validation(
    model_name="model_8_16_16_32_64_8_8_1_30_3_months_validation",
    model_history=history_8_16_16_32_64_8_8_1_30_3_months_validation,
)

Observations

Training loss and mean absolute error are decreasing gradually. Increasing the number of LSTM layers has not resulted in a decrease in mean absolute error when compared to one LSTM and one dense layer, which was trained for 50 epochs. To potentially further reduce the mean absolute error, increasing the epochs or number of LSTM layers and neurons may result in a decrease in MAE.

• 5 LSTM(8-16-16-32-64) 3 Dense(8-8-1)
• Epochs: 50
• Batch Size: 1
• Training Loss: 1.2129e-04
• Mean Absolute Error: 0.0068

Model Building with 5 LSTM and 3 Dense layers (8-16-16-32-64-8-8-1) and 30 epochs - 6 months

# Let's create a keras sequential model
model_8_16_16_32_64_8_8_1_30_6_months_validation = Sequential()

# Let's add two LSTM layer's to the model network
model_8_16_16_32_64_8_8_1_30_6_months_validation.add(
    LSTM(8, return_sequences=True, input_shape=(x_train_6_months.shape[1], 1))
)
model_8_16_16_32_64_8_8_1_30_6_months_validation.add(LSTM(16, return_sequences=True))
model_8_16_16_32_64_8_8_1_30_6_months_validation.add(LSTM(16, return_sequences=True))
model_8_16_16_32_64_8_8_1_30_6_months_validation.add(LSTM(32, return_sequences=True))
model_8_16_16_32_64_8_8_1_30_6_months_validation.add(LSTM(64, return_sequences=False))

# Let's add dense layer to the model network
model_8_16_16_32_64_8_8_1_30_6_months_validation.add(Dense(8))
model_8_16_16_32_64_8_8_1_30_6_months_validation.add(Dense(8))
model_8_16_16_32_64_8_8_1_30_6_months_validation.add(Dense(1))

# Let's build the model using mean_squared_error as the loss function
# and mean_absolute_error as the evaluation metric during the compilation process
model_8_16_16_32_64_8_8_1_30_6_months_validation.compile(
    optimizer="adam", loss="mean_squared_error", metrics=["mean_absolute_error"]
)

# Let's define the callbacks for Model saving and Early stopping
cb_check = keras.callbacks.ModelCheckpoint(
    filepath="model_8_16_16_32_64_8_8_1_30_6_months_validation_checkpoint_filepath",
    save_best_only=True,
    monitor="val_loss",
)

cb_early = keras.callbacks.EarlyStopping(monitor="val_loss", patience=30)

# Let's fit the sequential model with input features and output label
history_8_16_16_32_64_8_8_1_30_6_months_validation = (
    model_8_16_16_32_64_8_8_1_30_6_months_validation.fit(
        x_train_6_months,
        y_train_6_months,
        batch_size=16,
        epochs=30,
        callbacks=[cb_check, cb_early],
        validation_data=(x_valid_6_months, y_valid_6_months),
    )
)

Epoch 1/30
140/140 [==============================] - 22s 133ms/step - loss: 4.5942e-04 - mean_absolute_error: 0.0116 - val_loss: 9.1896e-04 - val_mean_absolute_error: 0.0265
Epoch 2/30
140/140 [==============================] - 14s 97ms/step - loss: 1.0051e-04 - mean_absolute_error: 0.0063 - val_loss: 9.2631e-04 - val_mean_absolute_error: 0.0265
Epoch 3/30
140/140 [==============================] - 14s 98ms/step - loss: 1.0511e-04 - mean_absolute_error: 0.0070 - val_loss: 0.0012 - val_mean_absolute_error: 0.0267
Epoch 4/30
140/140 [==============================] - 24s 172ms/step - loss: 8.3226e-05 - mean_absolute_error: 0.0059 - val_loss: 8.4088e-04 - val_mean_absolute_error: 0.0234
Epoch 5/30
140/140 [==============================] - 18s 129ms/step - loss: 8.9819e-05 - mean_absolute_error: 0.0062 - val_loss: 7.2047e-04 - val_mean_absolute_error: 0.0216
Epoch 6/30
140/140 [==============================] - 19s 133ms/step - loss: 7.4543e-05 - mean_absolute_error: 0.0058 - val_loss: 5.3191e-04 - val_mean_absolute_error: 0.0194
Epoch 7/30
140/140 [==============================] - 23s 161ms/step - loss: 6.2035e-05 - mean_absolute_error: 0.0048 - val_loss: 3.9165e-04 - val_mean_absolute_error: 0.0168
Epoch 8/30
140/140 [==============================] - 19s 134ms/step - loss: 5.8789e-05 - mean_absolute_error: 0.0052 - val_loss: 2.9001e-04 - val_mean_absolute_error: 0.0145
Epoch 9/30
140/140 [==============================] - 14s 98ms/step - loss: 4.1731e-05 - mean_absolute_error: 0.0042 - val_loss: 3.3276e-04 - val_mean_absolute_error: 0.0150
Epoch 10/30
140/140 [==============================] - 18s 126ms/step - loss: 3.5652e-05 - mean_absolute_error: 0.0038 - val_loss: 2.5492e-04 - val_mean_absolute_error: 0.0138
Epoch 11/30
140/140 [==============================] - 13s 93ms/step - loss: 4.3769e-05 - mean_absolute_error: 0.0044 - val_loss: 3.0692e-04 - val_mean_absolute_error: 0.0142
Epoch 12/30
140/140 [==============================] - 13s 94ms/step - loss: 5.3091e-05 - mean_absolute_error: 0.0050 - val_loss: 2.9345e-04 - val_mean_absolute_error: 0.0152
Epoch 13/30
140/140 [==============================] - 22s 155ms/step - loss: 4.0556e-05 - mean_absolute_error: 0.0044 - val_loss: 2.3248e-04 - val_mean_absolute_error: 0.0129
Epoch 14/30
140/140 [==============================] - 13s 96ms/step - loss: 3.9123e-05 - mean_absolute_error: 0.0042 - val_loss: 2.7823e-04 - val_mean_absolute_error: 0.0134
Epoch 15/30
140/140 [==============================] - 19s 134ms/step - loss: 4.0960e-05 - mean_absolute_error: 0.0045 - val_loss: 2.2997e-04 - val_mean_absolute_error: 0.0129
Epoch 16/30
140/140 [==============================] - 14s 97ms/step - loss: 3.3407e-05 - mean_absolute_error: 0.0039 - val_loss: 2.7271e-04 - val_mean_absolute_error: 0.0142
Epoch 17/30
140/140 [==============================] - 15s 104ms/step - loss: 3.3767e-05 - mean_absolute_error: 0.0039 - val_loss: 2.4817e-04 - val_mean_absolute_error: 0.0127
Epoch 18/30
140/140 [==============================] - 18s 131ms/step - loss: 3.1298e-05 - mean_absolute_error: 0.0039 - val_loss: 1.8788e-04 - val_mean_absolute_error: 0.0113
Epoch 19/30
140/140 [==============================] - 15s 105ms/step - loss: 3.3074e-05 - mean_absolute_error: 0.0038 - val_loss: 1.9666e-04 - val_mean_absolute_error: 0.0116
Epoch 20/30
140/140 [==============================] - 15s 105ms/step - loss: 2.7232e-05 - mean_absolute_error: 0.0034 - val_loss: 3.2547e-04 - val_mean_absolute_error: 0.0152
Epoch 21/30
140/140 [==============================] - 23s 168ms/step - loss: 4.5204e-05 - mean_absolute_error: 0.0047 - val_loss: 1.7645e-04 - val_mean_absolute_error: 0.0111
Epoch 22/30
140/140 [==============================] - 14s 102ms/step - loss: 4.3001e-05 - mean_absolute_error: 0.0047 - val_loss: 1.8177e-04 - val_mean_absolute_error: 0.0109
Epoch 23/30
140/140 [==============================] - 22s 154ms/step - loss: 2.5674e-05 - mean_absolute_error: 0.0032 - val_loss: 1.7145e-04 - val_mean_absolute_error: 0.0111
Epoch 24/30
140/140 [==============================] - 15s 104ms/step - loss: 2.6332e-05 - mean_absolute_error: 0.0035 - val_loss: 1.7803e-04 - val_mean_absolute_error: 0.0108
Epoch 25/30
140/140 [==============================] - 19s 137ms/step - loss: 2.9490e-05 - mean_absolute_error: 0.0038 - val_loss: 1.6480e-04 - val_mean_absolute_error: 0.0109
Epoch 26/30
140/140 [==============================] - 15s 104ms/step - loss: 3.3113e-05 - mean_absolute_error: 0.0039 - val_loss: 1.7382e-04 - val_mean_absolute_error: 0.0114
Epoch 27/30
140/140 [==============================] - 15s 105ms/step - loss: 3.2133e-05 - mean_absolute_error: 0.0039 - val_loss: 2.7129e-04 - val_mean_absolute_error: 0.0128
Epoch 28/30
140/140 [==============================] - 25s 176ms/step - loss: 3.3008e-05 - mean_absolute_error: 0.0041 - val_loss: 1.4723e-04 - val_mean_absolute_error: 0.0099
Epoch 29/30
140/140 [==============================] - 14s 98ms/step - loss: 2.1552e-05 - mean_absolute_error: 0.0031 - val_loss: 1.5715e-04 - val_mean_absolute_error: 0.0107
Epoch 30/30
140/140 [==============================] - 20s 141ms/step - loss: 2.1829e-05 - mean_absolute_error: 0.0032 - val_loss: 1.3055e-04 - val_mean_absolute_error: 0.0093

# Let's plot the mean_squared_error and mean_absolute_error using model history on the entire dataset
plot_loss_and_mae_validation(
    model_name="model_8_16_16_32_64_8_8_1_30_6_months_validation",
    model_history=history_8_16_16_32_64_8_8_1_30_6_months_validation,
)

Observations

Training loss and mean absolute error are decreasing gradually. Increasing the number of LSTM layers has not resulted in a decrease in mean absolute error when compared to one LSTM and one dense layer, which was trained for 50 epochs. To potentially further reduce the mean absolute error, increasing the epochs or number of LSTM layers and neurons may result in a decrease in MAE.

• 5 LSTM(8-16-16-32-64) 3 Dense(8-8-1)
• Epochs: 50
• Batch Size: 1
• Training Loss: 1.2129e-04
• Mean Absolute Error: 0.0068

Model Building with 6 LSTM and 2 Dense layers(16-32-32-32-64-64-8-1) and 30 epochs - 3 months

# Let's create a keras sequential model
model_16_32_32_32_64_64_8_1_30_3_months_validation = Sequential()

# Let's add two LSTM layer's to the model network
model_16_32_32_32_64_64_8_1_30_3_months_validation.add(
    LSTM(16, return_sequences=True, input_shape=(x_train_3_months.shape[1], 1))
)
model_16_32_32_32_64_64_8_1_30_3_months_validation.add(LSTM(32, return_sequences=True))
model_16_32_32_32_64_64_8_1_30_3_months_validation.add(LSTM(32, return_sequences=True))
model_16_32_32_32_64_64_8_1_30_3_months_validation.add(LSTM(32, return_sequences=True))
model_16_32_32_32_64_64_8_1_30_3_months_validation.add(LSTM(64, return_sequences=True))
model_16_32_32_32_64_64_8_1_30_3_months_validation.add(LSTM(64, return_sequences=False))


# Let's add dense layer to the model network
model_16_32_32_32_64_64_8_1_30_3_months_validation.add(Dense(8))
model_16_32_32_32_64_64_8_1_30_3_months_validation.add(Dense(1))

# Let's build the model using mean_squared_error as the loss function
# and mean_absolute_error as the evaluation metric during the compilation process
model_16_32_32_32_64_64_8_1_30_3_months_validation.compile(
    optimizer="adam", loss="mean_squared_error", metrics=["mean_absolute_error"]
)

# Let's define the callbacks for Model saving and Early stopping
cb_check = keras.callbacks.ModelCheckpoint(
    filepath="model_16_32_32_32_64_64_8_1_30_3_months_validation_checkpoint_filepath",
    save_best_only=True,
    monitor="val_loss",
)

cb_early = keras.callbacks.EarlyStopping(monitor="val_loss", patience=30)

# Let's fit the sequential model with input features and output label
history_16_32_32_32_64_64_8_1_30_3_months_validation = (
    model_16_32_32_32_64_64_8_1_30_3_months_validation.fit(
        x_train_3_months,
        y_train_3_months,
        batch_size=16,
        epochs=30,
        callbacks=[cb_check, cb_early],
        validation_data=(x_valid_3_months, y_valid_3_months),
    )
)

Epoch 1/30
146/146 [==============================] - 22s 115ms/step - loss: 4.9025e-04 - mean_absolute_error: 0.0128 - val_loss: 0.0052 - val_mean_absolute_error: 0.0573
Epoch 2/30
146/146 [==============================] - 20s 136ms/step - loss: 1.3975e-04 - mean_absolute_error: 0.0075 - val_loss: 0.0015 - val_mean_absolute_error: 0.0320
Epoch 3/30
146/146 [==============================] - 16s 109ms/step - loss: 1.2611e-04 - mean_absolute_error: 0.0070 - val_loss: 0.0012 - val_mean_absolute_error: 0.0293
Epoch 4/30
146/146 [==============================] - 11s 75ms/step - loss: 1.7005e-04 - mean_absolute_error: 0.0087 - val_loss: 0.0024 - val_mean_absolute_error: 0.0381
Epoch 5/30
146/146 [==============================] - 10s 70ms/step - loss: 8.2528e-05 - mean_absolute_error: 0.0056 - val_loss: 0.0012 - val_mean_absolute_error: 0.0285
Epoch 6/30
146/146 [==============================] - 16s 107ms/step - loss: 8.7561e-05 - mean_absolute_error: 0.0056 - val_loss: 9.8988e-04 - val_mean_absolute_error: 0.0256
Epoch 7/30
146/146 [==============================] - 19s 127ms/step - loss: 7.0223e-05 - mean_absolute_error: 0.0055 - val_loss: 6.7786e-04 - val_mean_absolute_error: 0.0214
Epoch 8/30
146/146 [==============================] - 10s 67ms/step - loss: 1.0240e-04 - mean_absolute_error: 0.0068 - val_loss: 0.0010 - val_mean_absolute_error: 0.0267
Epoch 9/30
146/146 [==============================] - 15s 103ms/step - loss: 6.2183e-05 - mean_absolute_error: 0.0049 - val_loss: 5.8524e-04 - val_mean_absolute_error: 0.0194
Epoch 10/30
146/146 [==============================] - 19s 127ms/step - loss: 5.4249e-05 - mean_absolute_error: 0.0050 - val_loss: 3.6512e-04 - val_mean_absolute_error: 0.0159
Epoch 11/30
146/146 [==============================] - 10s 66ms/step - loss: 6.0003e-05 - mean_absolute_error: 0.0052 - val_loss: 8.7230e-04 - val_mean_absolute_error: 0.0238
Epoch 12/30
146/146 [==============================] - 15s 104ms/step - loss: 4.1381e-05 - mean_absolute_error: 0.0041 - val_loss: 3.3797e-04 - val_mean_absolute_error: 0.0154
Epoch 13/30
146/146 [==============================] - 10s 66ms/step - loss: 4.2698e-05 - mean_absolute_error: 0.0043 - val_loss: 4.7989e-04 - val_mean_absolute_error: 0.0170
Epoch 14/30
146/146 [==============================] - 10s 67ms/step - loss: 4.3563e-05 - mean_absolute_error: 0.0043 - val_loss: 4.5835e-04 - val_mean_absolute_error: 0.0169
Epoch 15/30
146/146 [==============================] - 10s 65ms/step - loss: 4.6535e-05 - mean_absolute_error: 0.0047 - val_loss: 5.4940e-04 - val_mean_absolute_error: 0.0188
Epoch 16/30
146/146 [==============================] - 10s 66ms/step - loss: 3.8824e-05 - mean_absolute_error: 0.0039 - val_loss: 6.1487e-04 - val_mean_absolute_error: 0.0200
Epoch 17/30
146/146 [==============================] - 10s 67ms/step - loss: 4.4571e-05 - mean_absolute_error: 0.0046 - val_loss: 6.9617e-04 - val_mean_absolute_error: 0.0218
Epoch 18/30
146/146 [==============================] - 15s 103ms/step - loss: 3.6725e-05 - mean_absolute_error: 0.0040 - val_loss: 3.0598e-04 - val_mean_absolute_error: 0.0135
Epoch 19/30
146/146 [==============================] - 22s 150ms/step - loss: 4.8163e-05 - mean_absolute_error: 0.0046 - val_loss: 2.5709e-04 - val_mean_absolute_error: 0.0135
Epoch 20/30
146/146 [==============================] - 10s 68ms/step - loss: 3.9043e-05 - mean_absolute_error: 0.0041 - val_loss: 2.8349e-04 - val_mean_absolute_error: 0.0133
Epoch 21/30
146/146 [==============================] - 10s 69ms/step - loss: 3.9889e-05 - mean_absolute_error: 0.0040 - val_loss: 3.9686e-04 - val_mean_absolute_error: 0.0157
Epoch 22/30
146/146 [==============================] - 10s 66ms/step - loss: 4.1424e-05 - mean_absolute_error: 0.0043 - val_loss: 5.6669e-04 - val_mean_absolute_error: 0.0176
Epoch 23/30
146/146 [==============================] - 10s 68ms/step - loss: 3.6444e-05 - mean_absolute_error: 0.0041 - val_loss: 2.6650e-04 - val_mean_absolute_error: 0.0138
Epoch 24/30
146/146 [==============================] - 10s 67ms/step - loss: 5.3780e-05 - mean_absolute_error: 0.0046 - val_loss: 3.0853e-04 - val_mean_absolute_error: 0.0145
Epoch 25/30
146/146 [==============================] - 16s 107ms/step - loss: 2.6621e-05 - mean_absolute_error: 0.0033 - val_loss: 1.9130e-04 - val_mean_absolute_error: 0.0108
Epoch 26/30
146/146 [==============================] - 10s 66ms/step - loss: 2.8676e-05 - mean_absolute_error: 0.0035 - val_loss: 2.4982e-04 - val_mean_absolute_error: 0.0124
Epoch 27/30
146/146 [==============================] - 10s 67ms/step - loss: 2.3878e-05 - mean_absolute_error: 0.0032 - val_loss: 2.3682e-04 - val_mean_absolute_error: 0.0121
Epoch 28/30
146/146 [==============================] - 15s 101ms/step - loss: 2.4835e-05 - mean_absolute_error: 0.0033 - val_loss: 1.6591e-04 - val_mean_absolute_error: 0.0105
Epoch 29/30
146/146 [==============================] - 10s 66ms/step - loss: 2.9326e-05 - mean_absolute_error: 0.0036 - val_loss: 1.6714e-04 - val_mean_absolute_error: 0.0107
Epoch 30/30
146/146 [==============================] - 10s 67ms/step - loss: 2.3827e-05 - mean_absolute_error: 0.0031 - val_loss: 2.0104e-04 - val_mean_absolute_error: 0.0110

# Let's plot the mean_squared_error and mean_absolute_error using model history on the entire dataset
plot_loss_and_mae_validation(
    model_name="model_16_32_32_32_64_64_8_1_30_3_months_validation",
    model_history=history_16_32_32_32_64_64_8_1_30_3_months_validation,
)

Observations

Increasing the number of LSTM layers from five to six for 50 epochs, while reducing the number of dense layers from three to two, has not resulted in a decrease in the mean absolute error. Further steps, including increasing the number of epochs during training, might further reduce the mean absolute error.

• Epochs: 50
• Batch Size: 1
• Training Loss: 1.4036e-04
• Mean Absolute Error: 0.0076

Model Building with 6 LSTM and 2 Dense layers(16-32-32-32-64-64-8-1) and 30 epochs - 6 months

# Let's create a keras sequential model
model_16_32_32_32_64_64_8_1_30_6_months_validation = Sequential()

# Let's add two LSTM layer's to the model network
model_16_32_32_32_64_64_8_1_30_6_months_validation.add(
    LSTM(16, return_sequences=True, input_shape=(x_train_6_months.shape[1], 1))
)
model_16_32_32_32_64_64_8_1_30_6_months_validation.add(LSTM(32, return_sequences=True))
model_16_32_32_32_64_64_8_1_30_6_months_validation.add(LSTM(32, return_sequences=True))
model_16_32_32_32_64_64_8_1_30_6_months_validation.add(LSTM(32, return_sequences=True))
model_16_32_32_32_64_64_8_1_30_6_months_validation.add(LSTM(64, return_sequences=True))
model_16_32_32_32_64_64_8_1_30_6_months_validation.add(LSTM(64, return_sequences=False))


# Let's add dense layer to the model network
model_16_32_32_32_64_64_8_1_30_6_months_validation.add(Dense(8))
model_16_32_32_32_64_64_8_1_30_6_months_validation.add(Dense(1))

# Let's build the model using mean_squared_error as the loss function
# and mean_absolute_error as the evaluation metric during the compilation process
model_16_32_32_32_64_64_8_1_30_6_months_validation.compile(
    optimizer="adam", loss="mean_squared_error", metrics=["mean_absolute_error"]
)

# Let's define the callbacks for Model saving and Early stopping
cb_check = keras.callbacks.ModelCheckpoint(
    filepath="model_16_32_32_32_64_64_8_1_30_6_months_validation_checkpoint_filepath",
    save_best_only=True,
    monitor="val_loss",
)

cb_early = keras.callbacks.EarlyStopping(monitor="val_loss", patience=30)

# Let's fit the sequential model with input features and output label
history_16_32_32_32_64_64_8_1_30_6_months_validation = (
    model_16_32_32_32_64_64_8_1_30_6_months_validation.fit(
        x_train_6_months,
        y_train_6_months,
        batch_size=16,
        epochs=30,
        callbacks=[cb_check, cb_early],
        validation_data=(x_valid_6_months, y_valid_6_months),
    )
)

Epoch 1/30
140/140 [==============================] - 30s 167ms/step - loss: 3.8807e-04 - mean_absolute_error: 0.0116 - val_loss: 0.0011 - val_mean_absolute_error: 0.0292
Epoch 2/30
140/140 [==============================] - 18s 130ms/step - loss: 1.1471e-04 - mean_absolute_error: 0.0065 - val_loss: 0.0016 - val_mean_absolute_error: 0.0319
Epoch 3/30
140/140 [==============================] - 24s 169ms/step - loss: 1.2291e-04 - mean_absolute_error: 0.0071 - val_loss: 9.5226e-04 - val_mean_absolute_error: 0.0267
Epoch 4/30
140/140 [==============================] - 18s 130ms/step - loss: 1.0568e-04 - mean_absolute_error: 0.0066 - val_loss: 0.0010 - val_mean_absolute_error: 0.0269
Epoch 5/30
140/140 [==============================] - 27s 191ms/step - loss: 1.2108e-04 - mean_absolute_error: 0.0076 - val_loss: 8.2395e-04 - val_mean_absolute_error: 0.0240
Epoch 6/30
140/140 [==============================] - 24s 168ms/step - loss: 9.8437e-05 - mean_absolute_error: 0.0063 - val_loss: 6.3664e-04 - val_mean_absolute_error: 0.0215
Epoch 7/30
140/140 [==============================] - 19s 132ms/step - loss: 1.2136e-04 - mean_absolute_error: 0.0074 - val_loss: 6.8454e-04 - val_mean_absolute_error: 0.0223
Epoch 8/30
140/140 [==============================] - 23s 168ms/step - loss: 7.5423e-05 - mean_absolute_error: 0.0054 - val_loss: 5.8366e-04 - val_mean_absolute_error: 0.0191
Epoch 9/30
140/140 [==============================] - 26s 188ms/step - loss: 5.1169e-05 - mean_absolute_error: 0.0046 - val_loss: 4.2116e-04 - val_mean_absolute_error: 0.0166
Epoch 10/30
140/140 [==============================] - 18s 131ms/step - loss: 4.8586e-05 - mean_absolute_error: 0.0045 - val_loss: 5.1179e-04 - val_mean_absolute_error: 0.0186
Epoch 11/30
140/140 [==============================] - 23s 164ms/step - loss: 5.1840e-05 - mean_absolute_error: 0.0047 - val_loss: 3.0114e-04 - val_mean_absolute_error: 0.0149
Epoch 12/30
140/140 [==============================] - 18s 128ms/step - loss: 4.6481e-05 - mean_absolute_error: 0.0044 - val_loss: 4.2599e-04 - val_mean_absolute_error: 0.0164
Epoch 13/30
140/140 [==============================] - 18s 130ms/step - loss: 4.1785e-05 - mean_absolute_error: 0.0044 - val_loss: 4.5922e-04 - val_mean_absolute_error: 0.0168
Epoch 14/30
140/140 [==============================] - 24s 171ms/step - loss: 5.9786e-05 - mean_absolute_error: 0.0048 - val_loss: 2.6494e-04 - val_mean_absolute_error: 0.0138
Epoch 15/30
140/140 [==============================] - 20s 143ms/step - loss: 3.4139e-05 - mean_absolute_error: 0.0038 - val_loss: 3.0896e-04 - val_mean_absolute_error: 0.0145
Epoch 16/30
140/140 [==============================] - 18s 132ms/step - loss: 4.0099e-05 - mean_absolute_error: 0.0042 - val_loss: 2.8292e-04 - val_mean_absolute_error: 0.0147
Epoch 17/30
140/140 [==============================] - 18s 131ms/step - loss: 4.8800e-05 - mean_absolute_error: 0.0045 - val_loss: 5.6227e-04 - val_mean_absolute_error: 0.0193
Epoch 18/30
140/140 [==============================] - 19s 132ms/step - loss: 3.3254e-05 - mean_absolute_error: 0.0037 - val_loss: 2.7556e-04 - val_mean_absolute_error: 0.0145
Epoch 19/30
140/140 [==============================] - 18s 130ms/step - loss: 4.1551e-05 - mean_absolute_error: 0.0043 - val_loss: 9.1156e-04 - val_mean_absolute_error: 0.0258
Epoch 20/30
140/140 [==============================] - 18s 130ms/step - loss: 4.9028e-05 - mean_absolute_error: 0.0047 - val_loss: 2.8932e-04 - val_mean_absolute_error: 0.0145
Epoch 21/30
140/140 [==============================] - 31s 224ms/step - loss: 3.5930e-05 - mean_absolute_error: 0.0039 - val_loss: 2.6001e-04 - val_mean_absolute_error: 0.0139
Epoch 22/30
140/140 [==============================] - 24s 168ms/step - loss: 3.5292e-05 - mean_absolute_error: 0.0040 - val_loss: 2.5298e-04 - val_mean_absolute_error: 0.0134
Epoch 23/30
140/140 [==============================] - 20s 139ms/step - loss: 3.3611e-05 - mean_absolute_error: 0.0039 - val_loss: 2.8986e-04 - val_mean_absolute_error: 0.0142
Epoch 24/30
140/140 [==============================] - 25s 181ms/step - loss: 2.7646e-05 - mean_absolute_error: 0.0034 - val_loss: 2.1331e-04 - val_mean_absolute_error: 0.0124
Epoch 25/30
140/140 [==============================] - 30s 215ms/step - loss: 3.2739e-05 - mean_absolute_error: 0.0037 - val_loss: 1.7204e-04 - val_mean_absolute_error: 0.0109
Epoch 26/30
140/140 [==============================] - 25s 175ms/step - loss: 2.8030e-05 - mean_absolute_error: 0.0037 - val_loss: 1.5906e-04 - val_mean_absolute_error: 0.0102
Epoch 27/30
140/140 [==============================] - 25s 175ms/step - loss: 2.9130e-05 - mean_absolute_error: 0.0036 - val_loss: 1.4943e-04 - val_mean_absolute_error: 0.0102
Epoch 28/30
140/140 [==============================] - 19s 135ms/step - loss: 3.5492e-05 - mean_absolute_error: 0.0039 - val_loss: 2.3446e-04 - val_mean_absolute_error: 0.0129
Epoch 29/30
140/140 [==============================] - 29s 209ms/step - loss: 2.9636e-05 - mean_absolute_error: 0.0038 - val_loss: 1.4422e-04 - val_mean_absolute_error: 0.0100
Epoch 30/30
140/140 [==============================] - 25s 175ms/step - loss: 2.6554e-05 - mean_absolute_error: 0.0034 - val_loss: 1.4315e-04 - val_mean_absolute_error: 0.0099

# Let's plot the mean_squared_error and mean_absolute_error using model history on the entire dataset
plot_loss_and_mae_validation(
    model_name="model_16_32_32_32_64_64_8_1_30_6_months_validation",
    model_history=history_16_32_32_32_64_64_8_1_30_6_months_validation,
)

Observations

Increasing the number of LSTM layers from five to six for 50 epochs, while reducing the number of dense layers from three to two, has not resulted in a decrease in the mean absolute error. Further steps, including increasing the number of epochs during training, might further reduce the mean absolute error.

• Epochs: 50
• Batch Size: 1
• Training Loss: 1.4036e-04
• Mean Absolute Error: 0.0076

Increasing number of neurons in LSTM layers

Model Building with 3LSTM and 2Dense layers(64-128-128-8-1) and 30 epochs - 3 months

# Let's create a keras sequential model
model_64_128_128_8_1_30_3_months_validation = Sequential()

# Let's add two LSTM layer's to the model network
model_64_128_128_8_1_30_3_months_validation.add(
    LSTM(64, return_sequences=True, input_shape=(x_train_3_months.shape[1], 1))
)
model_64_128_128_8_1_30_3_months_validation.add(LSTM(128, return_sequences=True))
model_64_128_128_8_1_30_3_months_validation.add(LSTM(128, return_sequences=False))

# Let's add dense layer to the model network
model_64_128_128_8_1_30_3_months_validation.add(Dense(8))
model_64_128_128_8_1_30_3_months_validation.add(Dense(1))

# Let's build the model using mean_squared_error as the loss function
# and mean_absolute_error as the evaluation metric during the compilation process
model_64_128_128_8_1_30_3_months_validation.compile(
    optimizer="adam", loss="mean_squared_error", metrics=["mean_absolute_error"]
)

# Let's define the callbacks for Model saving and Early stopping
cb_check = keras.callbacks.ModelCheckpoint(
    filepath="model_64_128_128_8_1_30_3_months_validation_checkpoint_filepath",
    save_best_only=True,
    monitor="val_loss",
)

cb_early = keras.callbacks.EarlyStopping(monitor="val_loss", patience=30)

# Let's fit the sequential model with input features and output label
history_64_128_128_8_1_30_3_months_validation = (
    model_64_128_128_8_1_30_3_months_validation.fit(
        x_train_3_months,
        y_train_3_months,
        batch_size=16,
        epochs=30,
        callbacks=[cb_check, cb_early],
        validation_data=(x_valid_3_months, y_valid_3_months),
    )
)

Epoch 1/30
146/146 [==============================] - 20s 119ms/step - loss: 1.8490e-04 - mean_absolute_error: 0.0075 - val_loss: 0.0013 - val_mean_absolute_error: 0.0298
Epoch 2/30
146/146 [==============================] - 21s 141ms/step - loss: 6.2841e-05 - mean_absolute_error: 0.0051 - val_loss: 3.9742e-04 - val_mean_absolute_error: 0.0157
Epoch 3/30
146/146 [==============================] - 15s 105ms/step - loss: 3.8453e-05 - mean_absolute_error: 0.0041 - val_loss: 3.4054e-04 - val_mean_absolute_error: 0.0152
Epoch 4/30
146/146 [==============================] - 15s 103ms/step - loss: 4.0289e-05 - mean_absolute_error: 0.0043 - val_loss: 1.7781e-04 - val_mean_absolute_error: 0.0110
Epoch 5/30
146/146 [==============================] - 15s 104ms/step - loss: 3.2728e-05 - mean_absolute_error: 0.0038 - val_loss: 1.5763e-04 - val_mean_absolute_error: 0.0103
Epoch 6/30
146/146 [==============================] - 12s 85ms/step - loss: 2.7713e-05 - mean_absolute_error: 0.0034 - val_loss: 1.6545e-04 - val_mean_absolute_error: 0.0102
Epoch 7/30
146/146 [==============================] - 12s 84ms/step - loss: 2.7719e-05 - mean_absolute_error: 0.0035 - val_loss: 2.3832e-04 - val_mean_absolute_error: 0.0120
Epoch 8/30
146/146 [==============================] - 12s 84ms/step - loss: 3.4178e-05 - mean_absolute_error: 0.0038 - val_loss: 8.9350e-04 - val_mean_absolute_error: 0.0275
Epoch 9/30
146/146 [==============================] - 15s 103ms/step - loss: 6.5410e-05 - mean_absolute_error: 0.0053 - val_loss: 1.4138e-04 - val_mean_absolute_error: 0.0097
Epoch 10/30
146/146 [==============================] - 17s 116ms/step - loss: 1.6427e-05 - mean_absolute_error: 0.0026 - val_loss: 1.3151e-04 - val_mean_absolute_error: 0.0092
Epoch 11/30
146/146 [==============================] - 12s 82ms/step - loss: 1.9809e-05 - mean_absolute_error: 0.0029 - val_loss: 1.8615e-04 - val_mean_absolute_error: 0.0105
Epoch 12/30
146/146 [==============================] - 15s 105ms/step - loss: 3.2855e-05 - mean_absolute_error: 0.0040 - val_loss: 1.2252e-04 - val_mean_absolute_error: 0.0089
Epoch 13/30
146/146 [==============================] - 15s 101ms/step - loss: 2.2127e-05 - mean_absolute_error: 0.0031 - val_loss: 1.2009e-04 - val_mean_absolute_error: 0.0088
Epoch 14/30
146/146 [==============================] - 15s 101ms/step - loss: 1.5744e-05 - mean_absolute_error: 0.0026 - val_loss: 1.0231e-04 - val_mean_absolute_error: 0.0085
Epoch 15/30
146/146 [==============================] - 12s 84ms/step - loss: 1.4157e-05 - mean_absolute_error: 0.0024 - val_loss: 2.4802e-04 - val_mean_absolute_error: 0.0127
Epoch 16/30
146/146 [==============================] - 15s 101ms/step - loss: 2.4948e-05 - mean_absolute_error: 0.0034 - val_loss: 9.6328e-05 - val_mean_absolute_error: 0.0080
Epoch 17/30
146/146 [==============================] - 13s 89ms/step - loss: 1.8451e-05 - mean_absolute_error: 0.0028 - val_loss: 1.1609e-04 - val_mean_absolute_error: 0.0085
Epoch 18/30
146/146 [==============================] - 13s 88ms/step - loss: 2.3943e-05 - mean_absolute_error: 0.0033 - val_loss: 1.4565e-04 - val_mean_absolute_error: 0.0095
Epoch 19/30
146/146 [==============================] - 13s 89ms/step - loss: 1.4176e-05 - mean_absolute_error: 0.0024 - val_loss: 1.0301e-04 - val_mean_absolute_error: 0.0080
Epoch 20/30
146/146 [==============================] - 13s 89ms/step - loss: 2.3185e-05 - mean_absolute_error: 0.0033 - val_loss: 1.0679e-04 - val_mean_absolute_error: 0.0085
Epoch 21/30
146/146 [==============================] - 13s 89ms/step - loss: 1.7750e-05 - mean_absolute_error: 0.0029 - val_loss: 1.0756e-04 - val_mean_absolute_error: 0.0081
Epoch 22/30
146/146 [==============================] - 13s 90ms/step - loss: 1.5756e-05 - mean_absolute_error: 0.0026 - val_loss: 1.7864e-04 - val_mean_absolute_error: 0.0108
Epoch 23/30
146/146 [==============================] - 13s 91ms/step - loss: 1.5692e-05 - mean_absolute_error: 0.0027 - val_loss: 1.1011e-04 - val_mean_absolute_error: 0.0083
Epoch 24/30
146/146 [==============================] - 16s 109ms/step - loss: 1.3736e-05 - mean_absolute_error: 0.0025 - val_loss: 7.3034e-05 - val_mean_absolute_error: 0.0069
Epoch 25/30
146/146 [==============================] - 13s 88ms/step - loss: 2.6911e-05 - mean_absolute_error: 0.0038 - val_loss: 4.6336e-04 - val_mean_absolute_error: 0.0195
Epoch 26/30
146/146 [==============================] - 21s 147ms/step - loss: 1.1905e-05 - mean_absolute_error: 0.0023 - val_loss: 6.5778e-05 - val_mean_absolute_error: 0.0066
Epoch 27/30
146/146 [==============================] - 12s 81ms/step - loss: 1.2387e-05 - mean_absolute_error: 0.0023 - val_loss: 6.7416e-05 - val_mean_absolute_error: 0.0066
Epoch 28/30
146/146 [==============================] - 15s 100ms/step - loss: 1.1638e-05 - mean_absolute_error: 0.0023 - val_loss: 6.1568e-05 - val_mean_absolute_error: 0.0063
Epoch 29/30
146/146 [==============================] - 12s 82ms/step - loss: 1.2106e-05 - mean_absolute_error: 0.0022 - val_loss: 6.3776e-05 - val_mean_absolute_error: 0.0063
Epoch 30/30
146/146 [==============================] - 12s 81ms/step - loss: 1.3112e-05 - mean_absolute_error: 0.0023 - val_loss: 1.0106e-04 - val_mean_absolute_error: 0.0080

# Let's plot the mean_squared_error and mean_absolute_error using model history on the entire dataset
plot_loss_and_mae_validation(
    model_name="model_64_128_128_8_1_30_3_months_validation",
    model_history=history_64_128_128_8_1_30_3_months_validation,
)

Observations

Increasing the number of LSTM layers from five to six for 50 epochs, while reducing the number of dense layers from three to two, has not resulted in a decrease in the mean absolute error. Further steps, including increasing the number of epochs during training, might further reduce the mean absolute error.

• Epochs: 50
• Batch Size: 1
• Training Loss: 1.4036e-04
• Mean Absolute Error: 0.0076

Model Building with 3LSTM and 2Dense layers(64-128-128-8-1) and 30 epochs - 6 months

# Let's create a keras sequential model
model_64_128_128_8_1_30_6_months_validation = Sequential()

# Let's add two LSTM layer's to the model network
model_64_128_128_8_1_30_6_months_validation.add(
    LSTM(64, return_sequences=True, input_shape=(x_train_6_months.shape[1], 1))
)
model_64_128_128_8_1_30_6_months_validation.add(LSTM(128, return_sequences=True))
model_64_128_128_8_1_30_6_months_validation.add(LSTM(128, return_sequences=False))

# Let's add dense layer to the model network
model_64_128_128_8_1_30_6_months_validation.add(Dense(8))
model_64_128_128_8_1_30_6_months_validation.add(Dense(1))

# Let's build the model using mean_squared_error as the loss function
# and mean_absolute_error as the evaluation metric during the compilation process
model_64_128_128_8_1_30_6_months_validation.compile(
    optimizer="adam", loss="mean_squared_error", metrics=["mean_absolute_error"]
)

# Let's define the callbacks for Model saving and Early stopping
cb_check = keras.callbacks.ModelCheckpoint(
    filepath="model_64_128_128_8_1_30_6_months_validation_checkpoint_filepath",
    save_best_only=True,
    monitor="val_loss",
)

cb_early = keras.callbacks.EarlyStopping(monitor="val_loss", patience=30)

# Let's fit the sequential model with input features and output label
history_64_128_128_8_1_30_6_months_validation = (
    model_64_128_128_8_1_30_6_months_validation.fit(
        x_train_6_months,
        y_train_6_months,
        batch_size=16,
        epochs=30,
        callbacks=[cb_check, cb_early],
        validation_data=(x_valid_6_months, y_valid_6_months),
    )
)

Epoch 1/30
140/140 [==============================] - 29s 192ms/step - loss: 2.6542e-04 - mean_absolute_error: 0.0090 - val_loss: 5.1539e-04 - val_mean_absolute_error: 0.0177
Epoch 2/30
140/140 [==============================] - 26s 187ms/step - loss: 4.4961e-05 - mean_absolute_error: 0.0041 - val_loss: 3.4336e-04 - val_mean_absolute_error: 0.0147
Epoch 3/30
140/140 [==============================] - 26s 186ms/step - loss: 5.4879e-05 - mean_absolute_error: 0.0048 - val_loss: 2.5512e-04 - val_mean_absolute_error: 0.0129
Epoch 4/30
140/140 [==============================] - 33s 238ms/step - loss: 4.6260e-05 - mean_absolute_error: 0.0045 - val_loss: 2.1481e-04 - val_mean_absolute_error: 0.0121
Epoch 5/30
140/140 [==============================] - 27s 190ms/step - loss: 3.8005e-05 - mean_absolute_error: 0.0039 - val_loss: 1.6972e-04 - val_mean_absolute_error: 0.0107
Epoch 6/30
140/140 [==============================] - 22s 159ms/step - loss: 2.9176e-05 - mean_absolute_error: 0.0035 - val_loss: 2.0542e-04 - val_mean_absolute_error: 0.0113
Epoch 7/30
140/140 [==============================] - 26s 183ms/step - loss: 3.1865e-05 - mean_absolute_error: 0.0037 - val_loss: 1.3389e-04 - val_mean_absolute_error: 0.0094
Epoch 8/30
140/140 [==============================] - 25s 181ms/step - loss: 2.7342e-05 - mean_absolute_error: 0.0033 - val_loss: 1.2522e-04 - val_mean_absolute_error: 0.0092
Epoch 9/30
140/140 [==============================] - 26s 182ms/step - loss: 2.5762e-05 - mean_absolute_error: 0.0034 - val_loss: 1.1153e-04 - val_mean_absolute_error: 0.0089
Epoch 10/30
140/140 [==============================] - 23s 161ms/step - loss: 2.1304e-05 - mean_absolute_error: 0.0029 - val_loss: 1.7057e-04 - val_mean_absolute_error: 0.0104
Epoch 11/30
140/140 [==============================] - 22s 159ms/step - loss: 2.4943e-05 - mean_absolute_error: 0.0033 - val_loss: 1.7344e-04 - val_mean_absolute_error: 0.0105
Epoch 12/30
140/140 [==============================] - 26s 183ms/step - loss: 2.0270e-05 - mean_absolute_error: 0.0031 - val_loss: 8.8287e-05 - val_mean_absolute_error: 0.0080
Epoch 13/30
140/140 [==============================] - 22s 159ms/step - loss: 3.2210e-05 - mean_absolute_error: 0.0038 - val_loss: 1.3437e-04 - val_mean_absolute_error: 0.0092
Epoch 14/30
140/140 [==============================] - 22s 159ms/step - loss: 3.8696e-05 - mean_absolute_error: 0.0044 - val_loss: 1.0944e-04 - val_mean_absolute_error: 0.0084
Epoch 15/30
140/140 [==============================] - 23s 161ms/step - loss: 2.4318e-05 - mean_absolute_error: 0.0033 - val_loss: 1.4333e-04 - val_mean_absolute_error: 0.0099
Epoch 16/30
140/140 [==============================] - 34s 241ms/step - loss: 1.5933e-05 - mean_absolute_error: 0.0028 - val_loss: 8.6400e-05 - val_mean_absolute_error: 0.0079
Epoch 17/30
140/140 [==============================] - 23s 161ms/step - loss: 2.1326e-05 - mean_absolute_error: 0.0031 - val_loss: 1.2400e-04 - val_mean_absolute_error: 0.0090
Epoch 18/30
140/140 [==============================] - 26s 183ms/step - loss: 1.8458e-05 - mean_absolute_error: 0.0028 - val_loss: 7.6441e-05 - val_mean_absolute_error: 0.0073
Epoch 19/30
140/140 [==============================] - 22s 159ms/step - loss: 2.0338e-05 - mean_absolute_error: 0.0031 - val_loss: 8.5970e-05 - val_mean_absolute_error: 0.0075
Epoch 20/30
140/140 [==============================] - 23s 161ms/step - loss: 1.4304e-05 - mean_absolute_error: 0.0025 - val_loss: 1.0077e-04 - val_mean_absolute_error: 0.0078
Epoch 21/30
140/140 [==============================] - 23s 161ms/step - loss: 2.3257e-05 - mean_absolute_error: 0.0034 - val_loss: 8.2136e-05 - val_mean_absolute_error: 0.0074
Epoch 22/30
140/140 [==============================] - 22s 160ms/step - loss: 2.0076e-05 - mean_absolute_error: 0.0030 - val_loss: 8.7028e-05 - val_mean_absolute_error: 0.0073
Epoch 23/30
140/140 [==============================] - 22s 160ms/step - loss: 2.0926e-05 - mean_absolute_error: 0.0032 - val_loss: 9.9274e-05 - val_mean_absolute_error: 0.0079
Epoch 24/30
140/140 [==============================] - 23s 166ms/step - loss: 1.6506e-05 - mean_absolute_error: 0.0028 - val_loss: 8.7933e-05 - val_mean_absolute_error: 0.0074
Epoch 25/30
140/140 [==============================] - 23s 161ms/step - loss: 1.7069e-05 - mean_absolute_error: 0.0027 - val_loss: 1.0718e-04 - val_mean_absolute_error: 0.0084
Epoch 26/30
140/140 [==============================] - 26s 183ms/step - loss: 2.1527e-05 - mean_absolute_error: 0.0034 - val_loss: 6.5805e-05 - val_mean_absolute_error: 0.0065
Epoch 27/30
140/140 [==============================] - 26s 186ms/step - loss: 1.5568e-05 - mean_absolute_error: 0.0026 - val_loss: 5.4986e-05 - val_mean_absolute_error: 0.0061
Epoch 28/30
140/140 [==============================] - 23s 160ms/step - loss: 1.3820e-05 - mean_absolute_error: 0.0024 - val_loss: 5.5216e-05 - val_mean_absolute_error: 0.0061
Epoch 29/30
140/140 [==============================] - 26s 182ms/step - loss: 1.2934e-05 - mean_absolute_error: 0.0023 - val_loss: 5.4456e-05 - val_mean_absolute_error: 0.0060
Epoch 30/30
140/140 [==============================] - 23s 161ms/step - loss: 1.9919e-05 - mean_absolute_error: 0.0031 - val_loss: 9.6428e-05 - val_mean_absolute_error: 0.0079

# Let's plot the mean_squared_error and mean_absolute_error using model history on the entire dataset
plot_loss_and_mae_validation(
    model_name="model_64_128_128_8_1_30_6_months_validation",
    model_history=history_64_128_128_8_1_30_6_months_validation,
)

Observations

Increasing the number of LSTM layers from five to six for 50 epochs, while reducing the number of dense layers from three to two, has not resulted in a decrease in the mean absolute error. Further steps, including increasing the number of epochs during training, might further reduce the mean absolute error.

• Epochs: 50
• Batch Size: 1
• Training Loss: 1.4036e-04
• Mean Absolute Error: 0.0076

Model Building with 4 LSTM and 4 Dense layers (128-64-64-32-8-8-8-1) and 30 epochs - 3 months

# Let's create a keras sequential model
model_128_64_64_32_8_8_8_1_30_3_months_validation = Sequential()

# Let's add two LSTM layer's to the model network
model_128_64_64_32_8_8_8_1_30_3_months_validation.add(
    LSTM(128, return_sequences=True, input_shape=(x_train_3_months.shape[1], 1))
)
model_128_64_64_32_8_8_8_1_30_3_months_validation.add(LSTM(64, return_sequences=True))
model_128_64_64_32_8_8_8_1_30_3_months_validation.add(LSTM(64, return_sequences=True))
model_128_64_64_32_8_8_8_1_30_3_months_validation.add(LSTM(32, return_sequences=False))

# Let's add dense layer to the model network
model_128_64_64_32_8_8_8_1_30_3_months_validation.add(Dense(8))
model_128_64_64_32_8_8_8_1_30_3_months_validation.add(Dense(8))
model_128_64_64_32_8_8_8_1_30_3_months_validation.add(Dense(8))
model_128_64_64_32_8_8_8_1_30_3_months_validation.add(Dense(1))

# Let's build the model using mean_squared_error as the loss function
# and mean_absolute_error as the evaluation metric during the compilation process
model_128_64_64_32_8_8_8_1_30_3_months_validation.compile(
    optimizer="adam", loss="mean_squared_error", metrics=["mean_absolute_error"]
)

# Let's define the callbacks for Model saving and Early stopping
cb_check = keras.callbacks.ModelCheckpoint(
    filepath="model_128_64_64_32_8_8_8_1_30_3_months_validation_checkpoint_filepath",
    save_best_only=True,
    monitor="val_loss",
)

cb_early = keras.callbacks.EarlyStopping(monitor="val_loss", patience=30)

# Let's fit the sequential model with input features and output label
history_128_64_64_32_8_8_8_1_30_3_months_validation = (
    model_128_64_64_32_8_8_8_1_30_3_months_validation.fit(
        x_train_3_months,
        y_train_3_months,
        batch_size=16,
        epochs=30,
        callbacks=[cb_check, cb_early],
        validation_data=(x_valid_3_months, y_valid_3_months),
    )
)

Epoch 1/30
146/146 [==============================] - 23s 134ms/step - loss: 2.4304e-04 - mean_absolute_error: 0.0087 - val_loss: 0.0012 - val_mean_absolute_error: 0.0271
Epoch 2/30
146/146 [==============================] - 14s 94ms/step - loss: 1.0201e-04 - mean_absolute_error: 0.0069 - val_loss: 0.0011 - val_mean_absolute_error: 0.0256
Epoch 3/30
146/146 [==============================] - 14s 96ms/step - loss: 6.7097e-05 - mean_absolute_error: 0.0053 - val_loss: 3.9207e-04 - val_mean_absolute_error: 0.0166
Epoch 4/30
146/146 [==============================] - 14s 93ms/step - loss: 4.8645e-05 - mean_absolute_error: 0.0046 - val_loss: 3.6938e-04 - val_mean_absolute_error: 0.0160
Epoch 5/30
146/146 [==============================] - 10s 70ms/step - loss: 5.5090e-05 - mean_absolute_error: 0.0052 - val_loss: 4.3613e-04 - val_mean_absolute_error: 0.0166
Epoch 6/30
146/146 [==============================] - 10s 70ms/step - loss: 5.9716e-05 - mean_absolute_error: 0.0057 - val_loss: 4.5073e-04 - val_mean_absolute_error: 0.0171
Epoch 7/30
146/146 [==============================] - 14s 96ms/step - loss: 4.1688e-05 - mean_absolute_error: 0.0044 - val_loss: 2.2947e-04 - val_mean_absolute_error: 0.0117
Epoch 8/30
146/146 [==============================] - 11s 74ms/step - loss: 3.4830e-05 - mean_absolute_error: 0.0040 - val_loss: 2.9413e-04 - val_mean_absolute_error: 0.0133
Epoch 9/30
146/146 [==============================] - 17s 114ms/step - loss: 3.2743e-05 - mean_absolute_error: 0.0039 - val_loss: 2.2419e-04 - val_mean_absolute_error: 0.0116
Epoch 10/30
146/146 [==============================] - 10s 70ms/step - loss: 4.6830e-05 - mean_absolute_error: 0.0045 - val_loss: 3.5950e-04 - val_mean_absolute_error: 0.0149
Epoch 11/30
146/146 [==============================] - 10s 71ms/step - loss: 5.5157e-05 - mean_absolute_error: 0.0052 - val_loss: 2.2538e-04 - val_mean_absolute_error: 0.0118
Epoch 12/30
146/146 [==============================] - 10s 69ms/step - loss: 3.5803e-05 - mean_absolute_error: 0.0038 - val_loss: 3.3061e-04 - val_mean_absolute_error: 0.0148
Epoch 13/30
146/146 [==============================] - 11s 74ms/step - loss: 4.2138e-05 - mean_absolute_error: 0.0044 - val_loss: 8.2152e-04 - val_mean_absolute_error: 0.0253
Epoch 14/30
146/146 [==============================] - 10s 70ms/step - loss: 5.2816e-05 - mean_absolute_error: 0.0052 - val_loss: 8.1826e-04 - val_mean_absolute_error: 0.0253
Epoch 15/30
146/146 [==============================] - 14s 97ms/step - loss: 4.0779e-05 - mean_absolute_error: 0.0043 - val_loss: 1.4174e-04 - val_mean_absolute_error: 0.0096
Epoch 16/30
146/146 [==============================] - 14s 95ms/step - loss: 2.5635e-05 - mean_absolute_error: 0.0035 - val_loss: 1.4008e-04 - val_mean_absolute_error: 0.0096
Epoch 17/30
146/146 [==============================] - 14s 95ms/step - loss: 2.7556e-05 - mean_absolute_error: 0.0036 - val_loss: 1.3280e-04 - val_mean_absolute_error: 0.0095
Epoch 18/30
146/146 [==============================] - 11s 73ms/step - loss: 2.1464e-05 - mean_absolute_error: 0.0032 - val_loss: 1.4505e-04 - val_mean_absolute_error: 0.0097
Epoch 19/30
146/146 [==============================] - 10s 71ms/step - loss: 2.1680e-05 - mean_absolute_error: 0.0031 - val_loss: 1.3698e-04 - val_mean_absolute_error: 0.0097
Epoch 20/30
146/146 [==============================] - 10s 71ms/step - loss: 2.6265e-05 - mean_absolute_error: 0.0035 - val_loss: 1.4664e-04 - val_mean_absolute_error: 0.0097
Epoch 21/30
146/146 [==============================] - 10s 71ms/step - loss: 2.5490e-05 - mean_absolute_error: 0.0033 - val_loss: 1.9320e-04 - val_mean_absolute_error: 0.0110
Epoch 22/30
146/146 [==============================] - 10s 72ms/step - loss: 2.3375e-05 - mean_absolute_error: 0.0033 - val_loss: 3.6311e-04 - val_mean_absolute_error: 0.0161
Epoch 23/30
146/146 [==============================] - 19s 129ms/step - loss: 3.5493e-05 - mean_absolute_error: 0.0040 - val_loss: 1.0823e-04 - val_mean_absolute_error: 0.0085
Epoch 24/30
146/146 [==============================] - 14s 93ms/step - loss: 2.1631e-05 - mean_absolute_error: 0.0033 - val_loss: 1.0764e-04 - val_mean_absolute_error: 0.0086
Epoch 25/30
146/146 [==============================] - 10s 71ms/step - loss: 3.1187e-05 - mean_absolute_error: 0.0039 - val_loss: 2.0456e-04 - val_mean_absolute_error: 0.0113
Epoch 26/30
146/146 [==============================] - 10s 70ms/step - loss: 2.6626e-05 - mean_absolute_error: 0.0037 - val_loss: 1.2628e-04 - val_mean_absolute_error: 0.0093
Epoch 27/30
146/146 [==============================] - 10s 70ms/step - loss: 2.4198e-05 - mean_absolute_error: 0.0034 - val_loss: 6.0329e-04 - val_mean_absolute_error: 0.0222
Epoch 28/30
146/146 [==============================] - 14s 95ms/step - loss: 2.1241e-05 - mean_absolute_error: 0.0030 - val_loss: 8.7155e-05 - val_mean_absolute_error: 0.0077
Epoch 29/30
146/146 [==============================] - 10s 69ms/step - loss: 1.5903e-05 - mean_absolute_error: 0.0027 - val_loss: 1.2733e-04 - val_mean_absolute_error: 0.0090
Epoch 30/30
146/146 [==============================] - 14s 95ms/step - loss: 1.4886e-05 - mean_absolute_error: 0.0027 - val_loss: 7.9638e-05 - val_mean_absolute_error: 0.0074

# Let's plot the mean_squared_error and mean_absolute_error using model history on the entire dataset
plot_loss_and_mae_validation(
    model_name="model_128_64_64_32_8_8_8_1_30_3_months_validation",
    model_history=history_128_64_64_32_8_8_8_1_30_3_months_validation,
)

Observations

Training loss and mean absolute error are decreasing gradually. Increasing the number of Dense layers and neurons has not resulted in a decrease in mean absolute error when compared to one LSTM and one dense layer, which was trained for 50 epochs. To potentially further reduce the mean absolute error, increasing the number of LSTM layers and neurons may result in a decrease in MAE.

• 4 LSTM(8-16-16-32) 4 Dense(32-32-16-16-1)
• Epochs: 50
• Batch Size: 1
• Training Loss: 1.1192e-04
• Mean Absolute Error: 0.0070

Model Building with 4 LSTM and 4 Dense layers (128-64-64-32-8-8-8-1) and 30 epochs - 6 months

# Let's create a keras sequential model
model_128_64_64_32_8_8_8_1_30_6_months_validation = Sequential()

# Let's add two LSTM layer's to the model network
model_128_64_64_32_8_8_8_1_30_6_months_validation.add(
    LSTM(128, return_sequences=True, input_shape=(x_train_6_months.shape[1], 1))
)
model_128_64_64_32_8_8_8_1_30_6_months_validation.add(LSTM(64, return_sequences=True))
model_128_64_64_32_8_8_8_1_30_6_months_validation.add(LSTM(64, return_sequences=True))
model_128_64_64_32_8_8_8_1_30_6_months_validation.add(LSTM(32, return_sequences=False))

# Let's add dense layer to the model network
model_128_64_64_32_8_8_8_1_30_6_months_validation.add(Dense(8))
model_128_64_64_32_8_8_8_1_30_6_months_validation.add(Dense(8))
model_128_64_64_32_8_8_8_1_30_6_months_validation.add(Dense(8))
model_128_64_64_32_8_8_8_1_30_6_months_validation.add(Dense(1))

# Let's build the model using mean_squared_error as the loss function
# and mean_absolute_error as the evaluation metric during the compilation process
model_128_64_64_32_8_8_8_1_30_6_months_validation.compile(
    optimizer="adam", loss="mean_squared_error", metrics=["mean_absolute_error"]
)

# Let's define the callbacks for Model saving and Early stopping
cb_check = keras.callbacks.ModelCheckpoint(
    filepath="model_128_64_64_32_8_8_8_1_30_6_months_validation_checkpoint_filepath",
    save_best_only=True,
    monitor="val_loss",
)

cb_early = keras.callbacks.EarlyStopping(monitor="val_loss", patience=30)

# Let's fit the sequential model with input features and output label
history_128_64_64_32_8_8_8_1_30_6_months_validation = (
    model_128_64_64_32_8_8_8_1_30_6_months_validation.fit(
        x_train_6_months,
        y_train_6_months,
        batch_size=16,
        epochs=30,
        callbacks=[cb_check, cb_early],
        validation_data=(x_valid_6_months, y_valid_6_months),
    )
)

Epoch 1/30
140/140 [==============================] - 34s 208ms/step - loss: 3.4409e-04 - mean_absolute_error: 0.0092 - val_loss: 7.4808e-04 - val_mean_absolute_error: 0.0217
Epoch 2/30
140/140 [==============================] - 23s 161ms/step - loss: 8.0116e-05 - mean_absolute_error: 0.0059 - val_loss: 5.1701e-04 - val_mean_absolute_error: 0.0194
Epoch 3/30
140/140 [==============================] - 19s 136ms/step - loss: 6.7333e-05 - mean_absolute_error: 0.0052 - val_loss: 6.0306e-04 - val_mean_absolute_error: 0.0204
Epoch 4/30
140/140 [==============================] - 23s 164ms/step - loss: 7.9842e-05 - mean_absolute_error: 0.0061 - val_loss: 3.7977e-04 - val_mean_absolute_error: 0.0162
Epoch 5/30
140/140 [==============================] - 19s 136ms/step - loss: 6.5069e-05 - mean_absolute_error: 0.0053 - val_loss: 4.4539e-04 - val_mean_absolute_error: 0.0166
Epoch 6/30
140/140 [==============================] - 22s 161ms/step - loss: 5.1347e-05 - mean_absolute_error: 0.0047 - val_loss: 3.1168e-04 - val_mean_absolute_error: 0.0141
Epoch 7/30
140/140 [==============================] - 19s 138ms/step - loss: 7.0608e-05 - mean_absolute_error: 0.0060 - val_loss: 3.6302e-04 - val_mean_absolute_error: 0.0157
Epoch 8/30
140/140 [==============================] - 25s 179ms/step - loss: 4.4876e-05 - mean_absolute_error: 0.0044 - val_loss: 2.6663e-04 - val_mean_absolute_error: 0.0129
Epoch 9/30
140/140 [==============================] - 23s 164ms/step - loss: 2.7136e-05 - mean_absolute_error: 0.0034 - val_loss: 2.3670e-04 - val_mean_absolute_error: 0.0122
Epoch 10/30
140/140 [==============================] - 23s 165ms/step - loss: 2.8860e-05 - mean_absolute_error: 0.0036 - val_loss: 1.8598e-04 - val_mean_absolute_error: 0.0109
Epoch 11/30
140/140 [==============================] - 23s 167ms/step - loss: 3.6211e-05 - mean_absolute_error: 0.0041 - val_loss: 1.7303e-04 - val_mean_absolute_error: 0.0110
Epoch 12/30
140/140 [==============================] - 24s 171ms/step - loss: 4.5428e-05 - mean_absolute_error: 0.0049 - val_loss: 1.6257e-04 - val_mean_absolute_error: 0.0107
Epoch 13/30
140/140 [==============================] - 29s 208ms/step - loss: 3.2880e-05 - mean_absolute_error: 0.0038 - val_loss: 1.4806e-04 - val_mean_absolute_error: 0.0102
Epoch 14/30
140/140 [==============================] - 19s 137ms/step - loss: 3.0742e-05 - mean_absolute_error: 0.0036 - val_loss: 1.5622e-04 - val_mean_absolute_error: 0.0105
Epoch 15/30
140/140 [==============================] - 19s 135ms/step - loss: 2.7321e-05 - mean_absolute_error: 0.0037 - val_loss: 1.8513e-04 - val_mean_absolute_error: 0.0108
Epoch 16/30
140/140 [==============================] - 23s 162ms/step - loss: 4.7768e-05 - mean_absolute_error: 0.0052 - val_loss: 1.2106e-04 - val_mean_absolute_error: 0.0092
Epoch 17/30
140/140 [==============================] - 20s 139ms/step - loss: 2.4880e-05 - mean_absolute_error: 0.0034 - val_loss: 1.2140e-04 - val_mean_absolute_error: 0.0092
Epoch 18/30
140/140 [==============================] - 19s 137ms/step - loss: 3.0397e-05 - mean_absolute_error: 0.0039 - val_loss: 1.3773e-04 - val_mean_absolute_error: 0.0094
Epoch 19/30
140/140 [==============================] - 24s 169ms/step - loss: 3.3087e-05 - mean_absolute_error: 0.0040 - val_loss: 1.0772e-04 - val_mean_absolute_error: 0.0087
Epoch 20/30
140/140 [==============================] - 20s 140ms/step - loss: 2.1075e-05 - mean_absolute_error: 0.0031 - val_loss: 1.1225e-04 - val_mean_absolute_error: 0.0089
Epoch 21/30
140/140 [==============================] - 23s 168ms/step - loss: 3.0778e-05 - mean_absolute_error: 0.0038 - val_loss: 9.5077e-05 - val_mean_absolute_error: 0.0083
Epoch 22/30
140/140 [==============================] - 19s 137ms/step - loss: 2.7584e-05 - mean_absolute_error: 0.0036 - val_loss: 9.9600e-05 - val_mean_absolute_error: 0.0084
Epoch 23/30
140/140 [==============================] - 30s 216ms/step - loss: 2.2385e-05 - mean_absolute_error: 0.0032 - val_loss: 8.4882e-05 - val_mean_absolute_error: 0.0078
Epoch 24/30
140/140 [==============================] - 23s 163ms/step - loss: 1.9895e-05 - mean_absolute_error: 0.0030 - val_loss: 7.8880e-05 - val_mean_absolute_error: 0.0075
Epoch 25/30
140/140 [==============================] - 20s 144ms/step - loss: 2.1346e-05 - mean_absolute_error: 0.0032 - val_loss: 1.3067e-04 - val_mean_absolute_error: 0.0093
Epoch 26/30
140/140 [==============================] - 19s 137ms/step - loss: 1.7836e-05 - mean_absolute_error: 0.0028 - val_loss: 1.4322e-04 - val_mean_absolute_error: 0.0095
Epoch 27/30
140/140 [==============================] - 23s 165ms/step - loss: 3.4053e-05 - mean_absolute_error: 0.0042 - val_loss: 7.8772e-05 - val_mean_absolute_error: 0.0075
Epoch 28/30
140/140 [==============================] - 19s 137ms/step - loss: 2.4546e-05 - mean_absolute_error: 0.0033 - val_loss: 1.4337e-04 - val_mean_absolute_error: 0.0098
Epoch 29/30
140/140 [==============================] - 20s 139ms/step - loss: 3.7113e-05 - mean_absolute_error: 0.0041 - val_loss: 1.0836e-04 - val_mean_absolute_error: 0.0084
Epoch 30/30
140/140 [==============================] - 23s 165ms/step - loss: 2.0188e-05 - mean_absolute_error: 0.0030 - val_loss: 7.3160e-05 - val_mean_absolute_error: 0.0071

# Let's plot the mean_squared_error and mean_absolute_error using model history on the entire dataset
plot_loss_and_mae_validation(
    model_name="model_128_64_64_32_8_8_8_1_30_6_months_validation",
    model_history=history_128_64_64_32_8_8_8_1_30_6_months_validation,
)

Observations

Training loss and mean absolute error are decreasing gradually. Increasing the number of Dense layers and neurons has not resulted in a decrease in mean absolute error when compared to one LSTM and one dense layer, which was trained for 50 epochs. To potentially further reduce the mean absolute error, increasing the number of LSTM layers and neurons may result in a decrease in MAE.

• 4 LSTM(8-16-16-32) 4 Dense(32-32-16-16-1)
• Epochs: 50
• Batch Size: 1
• Training Loss: 1.1192e-04
• Mean Absolute Error: 0.0070

Prediction Accuracy Visual Overview

# Let's import scikit-learn to get the metrics
from sklearn.metrics import (
    mean_squared_error,
    mean_absolute_error,
    mean_absolute_percentage_error,
)

def calculate_metrics(actual_prices, predicted_prices):
    # mse = mean_squared_error(actual_prices, predicted_prices)
    mae = mean_absolute_error(actual_prices, predicted_prices)
    mape = mean_absolute_percentage_error(actual_prices, predicted_prices)
    return mae, mape

def get_metrics(period, model):
    # Let's get the actual_values and predictions out of the model
    if period == 3:
        valid_input_values = x_valid_3_months
        valid_output_values = [y_valid_3_months]
        test_input_values = x_test_3_months
        test_output_values = [y_test_3_months]

    elif period == 6:
        valid_input_values = x_valid_6_months
        valid_output_values = [y_valid_6_months]
        test_input_values = x_test_6_months
        test_output_values = [y_test_6_months]

    # Let's run preidtcions on validation and test data
    valid_predicted_prices_scaled = model.predict(valid_input_values, verbose=0)
    test_predicted_prices_scaled = model.predict(test_input_values, verbose=0)

    # Inverse the scaling to get actual price predictions
    valid_predicted_prices = scaler.inverse_transform(valid_predicted_prices_scaled)
    test_predicted_prices = scaler.inverse_transform(test_predicted_prices_scaled)

    # Ensure the actual test data is in the same shape and inverse scaled
    valid_actual_prices = scaler.inverse_transform(valid_output_values)
    test_actual_prices = scaler.inverse_transform(test_output_values)

    # Let's reduce the dimesnions of both scaled predictions and output values of validation data
    valid_actual_prices = np.squeeze(valid_actual_prices)
    valid_predicted_prices = np.squeeze(valid_predicted_prices)

    # Let's reduce the dimesnions of both scaled predictions and output values of test data
    test_actual_prices = np.squeeze(test_actual_prices)
    test_predicted_prices = np.squeeze(test_predicted_prices)

    validation_mae, validation_mape = calculate_metrics(
        valid_actual_prices, valid_predicted_prices
    )

    test_mae, test_mape = calculate_metrics(test_actual_prices, test_predicted_prices)

    metrics = {
        "valid_actual_prices": valid_actual_prices,
        "valid_predicted_prices": valid_predicted_prices,
        "test_actual_prices": test_actual_prices,
        "test_predicted_prices": test_predicted_prices,
        "validation_mae": validation_mae,
        "validation_mape": validation_mape,
        "test_mae": test_mae,
        "test_mape": test_mape,
    }

    return metrics

# Function to plot the test predictions and actual predictions
def plot_metrics_and_predictions(model_name, actual_prices, predicted_prices):
    figure, (ax1, ax2) = plt.subplots(
        1, 2, figsize=(18, 6), dpi=300
    )  # Define a figure with two subplots
    figure.suptitle(model_name)  # Set the title of the figure

    # Plot the true vs predicted values on the second subplot
    ax1.plot(
        actual_prices, label="Actual Values", color="green"
    )  # Plot the actual values
    ax1.plot(
        predicted_prices, label="Predicted Values", color="orange"
    )  # Plot the predicted values
    ax1.set_title(
        "Actual vs. Predicted Values"
    )  # Title for the true vs. predicted plot
    ax1.set_xlabel("Data Points")  # Label for the x-axis
    ax1.set_ylabel("Values")  # Label for the y-axis
    ax1.legend(loc="upper right")  # Legend positioned in the upper right

    ax2.scatter(
        actual_prices, predicted_prices, color="blue", s=10, alpha=0.5
    )  # Scatter plot of actual vs predicted
    ax2.set_title("Actual vs. Predicted Values")  # Title for the plot
    ax2.set_xlabel("Actual Values")  # Label for the x-axis (True Values)
    ax2.set_ylabel("Predicted Values")  # Label for the y-axis (Predicted Values)
    ax2.plot(
        [min(actual_prices), max(actual_prices)],
        [min(actual_prices), max(actual_prices)],
        "r--",
    )  # Red line showing perfect predictions

    # Add a legend to clarify the elements in the plot
    ax2.legend(["Predicted vs Actual", "Ideal Predictions"])
    plt.savefig("plots/best_regularization_model.png")
    plt.show()

# Let's define the list of models that are used for validation and comparison.
model_validation_list = {
    "model_8_1_30_3_months_validation": model_8_1_30_3_months_validation,
    "model_8_1_30_6_months_validation": model_8_1_30_6_months_validation,
    "model_64_1_30_3_months_validation": model_64_1_30_3_months_validation,
    "model_64_1_30_6_months_validation": model_64_1_30_6_months_validation,
    "model_128_1_30_3_months_validation": model_128_1_30_3_months_validation,
    "model_128_1_30_6_months_validation": model_128_1_30_6_months_validation,
    "model_128_64_8_1_30_3_months_validation": model_128_64_8_1_30_3_months_validation,
    "model_128_64_8_1_30_6_months_validation": model_128_64_8_1_30_6_months_validation,
    "model_128_64_64_8_8_1_30_3_months_validation": model_128_64_64_8_8_1_30_3_months_validation,
    "model_128_64_64_8_8_1_30_6_months_validation": model_128_64_64_8_8_1_30_6_months_validation,
    "model_8_16_16_32_8_8_1_30_3_months_validation": model_8_16_16_32_8_8_1_30_3_months_validation,
    "model_8_16_16_32_8_8_1_30_6_months_validation": model_8_16_16_32_8_8_1_30_6_months_validation,
    "model_8_16_16_32_32_16_16_1_30_3_months_validation": model_8_16_16_32_32_16_16_1_30_3_months_validation,
    "model_8_16_16_32_32_16_16_1_30_6_months_validation": model_8_16_16_32_32_16_16_1_30_6_months_validation,
    "model_8_16_16_32_64_8_8_1_30_3_months_validation": model_8_16_16_32_64_8_8_1_30_3_months_validation,
    "model_8_16_16_32_64_8_8_1_30_6_months_validation": model_8_16_16_32_64_8_8_1_30_6_months_validation,
    "model_16_32_32_32_64_64_8_1_30_3_months_validation": model_16_32_32_32_64_64_8_1_30_3_months_validation,
    "model_16_32_32_32_64_64_8_1_30_6_months_validation": model_16_32_32_32_64_64_8_1_30_6_months_validation,
    "model_64_128_128_8_1_30_3_months_validation": model_64_128_128_8_1_30_3_months_validation,
    "model_64_128_128_8_1_30_6_months_validation": model_64_128_128_8_1_30_6_months_validation,
    "model_128_64_64_32_8_8_8_1_30_3_months_validation": model_128_64_64_32_8_8_8_1_30_3_months_validation,
    "model_128_64_64_32_8_8_8_1_30_6_months_validation": model_128_64_64_32_8_8_8_1_30_6_months_validation,
}

3 Months Window

validation_testing_3_months = []

for model_name in model_validation_list.keys():
    model = keras.models.load_model(f"{model_name}_checkpoint_filepath")
    if "3_months" in model_name:
        three_months_metrics_dict = get_metrics(period=3, model=model)
        plot_metrics_and_predictions(
            model_name=model_name,
            actual_prices=three_months_metrics_dict.get("valid_actual_prices"),
            predicted_prices=three_months_metrics_dict.get("valid_predicted_prices"),
        )
        validation_testing_3_months.append(
            {
                "Model Configuration": model_name,
                "period": "3 months",
                "Validation MAE": three_months_metrics_dict.get("validation_mae"),
                "Validation MAPE": three_months_metrics_dict.get("validation_mape"),
            }
        )

Printing Validation MSE and MAE for all the models - 3 months

# Let's convert the validation_testing list of dictionaries into a pandas data_frame
validation_testing_3_months_df = pd.DataFrame(validation_testing_3_months)
validation_testing_3_months_df

	Model Configuration	period	Validation MAE	Validation MAPE
0	model_8_1_30_3_months_validation	3 months	7.214011	0.044383
1	model_64_1_30_3_months_validation	3 months	6.303906	0.038462
2	model_128_1_30_3_months_validation	3 months	6.152192	0.037394
3	model_128_64_8_1_30_3_months_validation	3 months	5.962914	0.035794
4	model_128_64_64_8_8_1_30_3_months_validation	3 months	6.445919	0.038820
5	model_8_16_16_32_8_8_1_30_3_months_validation	3 months	9.637526	0.059911
6	model_8_16_16_32_32_16_16_1_30_3_months_valida...	3 months	9.583087	0.059262
7	model_8_16_16_32_64_8_8_1_30_3_months_validation	3 months	11.107743	0.070483
8	model_16_32_32_32_64_64_8_1_30_3_months_valida...	3 months	9.933542	0.061433
9	model_64_128_128_8_1_30_3_months_validation	3 months	6.004318	0.036584
10	model_128_64_64_32_8_8_8_1_30_3_months_validation	3 months	7.009423	0.042522

6 months Window

# Let's define the list of models that are used for validation and comparison.
model_validation_test = {
    "model_128_64_64_8_8_1_30_6_months_validation": model_128_64_64_8_8_1_30_6_months_validation
}

validation_testing_6_months = []

for model_name in model_validation_test.keys():
    if "6_months" in model_name:
        model = keras.models.load_model(f"{model_name}_checkpoint_filepath")
        six_months_metrics_dict = get_metrics(period=6, model=model)
        plot_metrics_and_predictions(
            model_name=model_name,
            actual_prices=six_months_metrics_dict.get("valid_actual_prices"),
            predicted_prices=six_months_metrics_dict.get("valid_predicted_prices"),
        )
        validation_testing_6_months.append(
            {
                "Model Configuration": model_name,
                "period": "6 months",
                "Validation MAE": six_months_metrics_dict.get("validation_mae"),
                "Validation MAPE": six_months_metrics_dict.get("validation_mape"),
            }
        )

Printing Validation MSE and MAE for all the models - 6 months

# Let's convert the validation_testing list of dictionaries into a pandas data_frame
validation_testing_6_months_df = pd.DataFrame(validation_testing_6_months)
validation_testing_6_months_df

	Model Configuration	period	Validation MAE	Validation MAPE
0	model_8_1_30_6_months_validation	6 months	6.490327	0.042146
1	model_64_1_30_6_months_validation	6 months	5.766417	0.036950
2	model_128_1_30_6_months_validation	6 months	5.635041	0.036111
3	model_128_64_8_1_30_6_months_validation	6 months	5.801102	0.037494
4	model_128_64_64_8_8_1_30_6_months_validation	6 months	5.871108	0.038194
5	model_8_16_16_32_8_8_1_30_6_months_validation	6 months	9.925282	0.067345
6	model_8_16_16_32_32_16_16_1_30_6_months_valida...	6 months	10.011879	0.067544
7	model_8_16_16_32_64_8_8_1_30_6_months_validation	6 months	8.775115	0.059131
8	model_16_32_32_32_64_64_8_1_30_6_months_valida...	6 months	9.335995	0.063925
9	model_64_128_128_8_1_30_6_months_validation	6 months	5.637837	0.036104
10	model_128_64_64_32_8_8_8_1_30_6_months_validation	6 months	6.744692	0.043751

Observations

---

📊From the tables above, which show comparisons of different LSTM model configurations, each row represents a distinct configuration of LSTM layers and neurons. These models were trained considering either a 3-month or a 6-month window to predict daily stock prices. Comparing the two tables, it is evident that the LSTM models trained with a 6-month window outperformed those trained with a 3-month window in predicting future stock prices. 📈

💡The best model among the different configurations of LSTM layers and neurons is the one with three LSTM layers (128, 64, and 64 neurons, respectively) and three dense layers (8, 8, and 1 neurons, respectively).🏆

Data Augmentation

Jittering - 0.005

def add_noise(data, noise_level):
    # Generate random Gaussian noise. This noise has a mean of 0 and a standard deviation provided by 'noise_level'.
    # The size of the noise array is the same as the size of the input data.
    noise = np.random.normal(loc=0.0, scale=noise_level, size=data.shape)

    # Add the generated noise to the original data. This simulates real-world unpredictability in the data.
    return data + noise

# Let's add random noise to six-month training data window
x_train_6_months_noisy_0_0_0_5 = add_noise(x_train_6_months, noise_level=0.005)

Model Building with 3 LSTM and 3 Dense layers (128-64-64-8-8-1) and 30 epochs - 6 months

# Let's create a keras sequential model
model_128_64_64_8_8_1_30_6_months_validation_0_0_0_5_noise = Sequential()

# Let's add two LSTM layer's to the model network
model_128_64_64_8_8_1_30_6_months_validation_0_0_0_5_noise.add(
    LSTM(
        128,
        return_sequences=True,
        input_shape=(x_train_6_months_noisy_0_0_0_5.shape[1], 1),
    )
)
model_128_64_64_8_8_1_30_6_months_validation_0_0_0_5_noise.add(
    LSTM(64, return_sequences=True)
)
model_128_64_64_8_8_1_30_6_months_validation_0_0_0_5_noise.add(
    LSTM(64, return_sequences=False)
)

# Let's add dense layer to the model network
model_128_64_64_8_8_1_30_6_months_validation_0_0_0_5_noise.add(Dense(8))
model_128_64_64_8_8_1_30_6_months_validation_0_0_0_5_noise.add(Dense(8))
model_128_64_64_8_8_1_30_6_months_validation_0_0_0_5_noise.add(Dense(1))

# Let's build the model using mean_squared_error as the loss function
# and mean_absolute_error as the evaluation metric during the compilation process
model_128_64_64_8_8_1_30_6_months_validation_0_0_0_5_noise.compile(
    optimizer="adam", loss="mean_squared_error", metrics=["mean_absolute_error"]
)

# Let's define the callbacks for Model saving and Early stopping
cb_check = keras.callbacks.ModelCheckpoint(
    filepath="model_128_64_64_8_8_1_30_6_months_validation_0_0_0_5_noise_checkpoint_filepath",
    save_best_only=True,
    monitor="val_loss",
)

cb_early = keras.callbacks.EarlyStopping(monitor="val_loss", patience=30)

# Let's fit the sequential model with input features and output label
history_128_64_64_8_8_1_30_6_months_validation_0_0_0_5_noise = (
    model_128_64_64_8_8_1_30_6_months_validation_0_0_0_5_noise.fit(
        x_train_6_months_noisy_0_0_0_5,
        y_train_6_months,
        batch_size=16,
        epochs=30,
        callbacks=[cb_check, cb_early],
        validation_data=(x_valid_6_months, y_valid_6_months),
    )
)

Epoch 1/30
140/140 [==============================] - 38s 243ms/step - loss: 2.8165e-04 - mean_absolute_error: 0.0086 - val_loss: 4.3918e-04 - val_mean_absolute_error: 0.0178
Epoch 2/30
140/140 [==============================] - 32s 226ms/step - loss: 7.0703e-05 - mean_absolute_error: 0.0054 - val_loss: 3.9648e-04 - val_mean_absolute_error: 0.0157
Epoch 3/30
140/140 [==============================] - 39s 275ms/step - loss: 4.3766e-05 - mean_absolute_error: 0.0043 - val_loss: 2.8942e-04 - val_mean_absolute_error: 0.0142
Epoch 4/30
140/140 [==============================] - 27s 191ms/step - loss: 5.7341e-05 - mean_absolute_error: 0.0052 - val_loss: 3.2295e-04 - val_mean_absolute_error: 0.0147
Epoch 5/30
140/140 [==============================] - 44s 311ms/step - loss: 4.7397e-05 - mean_absolute_error: 0.0048 - val_loss: 2.1378e-04 - val_mean_absolute_error: 0.0119
Epoch 6/30
140/140 [==============================] - 28s 198ms/step - loss: 3.6544e-05 - mean_absolute_error: 0.0041 - val_loss: 2.2197e-04 - val_mean_absolute_error: 0.0122
Epoch 7/30
140/140 [==============================] - 38s 274ms/step - loss: 3.8310e-05 - mean_absolute_error: 0.0042 - val_loss: 1.9205e-04 - val_mean_absolute_error: 0.0111
Epoch 8/30
140/140 [==============================] - 56s 401ms/step - loss: 3.5402e-05 - mean_absolute_error: 0.0040 - val_loss: 1.8193e-04 - val_mean_absolute_error: 0.0109
Epoch 9/30
140/140 [==============================] - 40s 288ms/step - loss: 2.6038e-05 - mean_absolute_error: 0.0035 - val_loss: 1.4729e-04 - val_mean_absolute_error: 0.0097
Epoch 10/30
140/140 [==============================] - 39s 275ms/step - loss: 4.5284e-05 - mean_absolute_error: 0.0046 - val_loss: 1.3189e-04 - val_mean_absolute_error: 0.0094
Epoch 11/30
140/140 [==============================] - 29s 205ms/step - loss: 4.6096e-05 - mean_absolute_error: 0.0047 - val_loss: 3.1114e-04 - val_mean_absolute_error: 0.0142
Epoch 12/30
140/140 [==============================] - 31s 222ms/step - loss: 3.2182e-05 - mean_absolute_error: 0.0041 - val_loss: 1.1174e-04 - val_mean_absolute_error: 0.0088
Epoch 13/30
140/140 [==============================] - 26s 187ms/step - loss: 3.5503e-05 - mean_absolute_error: 0.0043 - val_loss: 1.9206e-04 - val_mean_absolute_error: 0.0114
Epoch 14/30
140/140 [==============================] - 26s 186ms/step - loss: 2.8943e-05 - mean_absolute_error: 0.0038 - val_loss: 1.1248e-04 - val_mean_absolute_error: 0.0088
Epoch 15/30
140/140 [==============================] - 26s 182ms/step - loss: 2.8993e-05 - mean_absolute_error: 0.0038 - val_loss: 1.2521e-04 - val_mean_absolute_error: 0.0089
Epoch 16/30
140/140 [==============================] - 29s 205ms/step - loss: 2.7478e-05 - mean_absolute_error: 0.0037 - val_loss: 1.5544e-04 - val_mean_absolute_error: 0.0103
Epoch 17/30
140/140 [==============================] - 34s 246ms/step - loss: 2.5139e-05 - mean_absolute_error: 0.0035 - val_loss: 9.1179e-05 - val_mean_absolute_error: 0.0081
Epoch 18/30
140/140 [==============================] - 34s 238ms/step - loss: 2.4997e-05 - mean_absolute_error: 0.0036 - val_loss: 8.8506e-05 - val_mean_absolute_error: 0.0079
Epoch 19/30
140/140 [==============================] - 34s 241ms/step - loss: 3.9542e-05 - mean_absolute_error: 0.0046 - val_loss: 1.1174e-04 - val_mean_absolute_error: 0.0087
Epoch 20/30
140/140 [==============================] - 28s 198ms/step - loss: 2.3729e-05 - mean_absolute_error: 0.0034 - val_loss: 1.2598e-04 - val_mean_absolute_error: 0.0088
Epoch 21/30
140/140 [==============================] - 34s 245ms/step - loss: 3.1113e-05 - mean_absolute_error: 0.0040 - val_loss: 9.4390e-05 - val_mean_absolute_error: 0.0078
Epoch 22/30
140/140 [==============================] - 18s 127ms/step - loss: 3.0985e-05 - mean_absolute_error: 0.0039 - val_loss: 9.5298e-05 - val_mean_absolute_error: 0.0080
Epoch 23/30
140/140 [==============================] - 24s 167ms/step - loss: 2.4779e-05 - mean_absolute_error: 0.0034 - val_loss: 8.7464e-05 - val_mean_absolute_error: 0.0078
Epoch 24/30
140/140 [==============================] - 16s 116ms/step - loss: 2.9571e-05 - mean_absolute_error: 0.0038 - val_loss: 1.4312e-04 - val_mean_absolute_error: 0.0097
Epoch 25/30
140/140 [==============================] - 17s 119ms/step - loss: 2.7821e-05 - mean_absolute_error: 0.0037 - val_loss: 9.2686e-05 - val_mean_absolute_error: 0.0077
Epoch 26/30
140/140 [==============================] - 17s 121ms/step - loss: 2.0445e-05 - mean_absolute_error: 0.0033 - val_loss: 1.1474e-04 - val_mean_absolute_error: 0.0087
Epoch 27/30
140/140 [==============================] - 21s 148ms/step - loss: 2.4047e-05 - mean_absolute_error: 0.0035 - val_loss: 7.5860e-05 - val_mean_absolute_error: 0.0073
Epoch 28/30
140/140 [==============================] - 18s 127ms/step - loss: 2.8001e-05 - mean_absolute_error: 0.0037 - val_loss: 9.0159e-05 - val_mean_absolute_error: 0.0078
Epoch 29/30
140/140 [==============================] - 16s 115ms/step - loss: 2.1817e-05 - mean_absolute_error: 0.0033 - val_loss: 8.5957e-05 - val_mean_absolute_error: 0.0076
Epoch 30/30
140/140 [==============================] - 19s 137ms/step - loss: 2.9440e-05 - mean_absolute_error: 0.0040 - val_loss: 7.4957e-05 - val_mean_absolute_error: 0.0072

# Let's plot the mean_squared_error and mean_absolute_error using model history on the entire dataset
plot_loss_and_mae_validation(
    model_name="model_128_64_64_8_8_1_30_6_months_validation_0_0_0_5_noise",
    model_history=history_128_64_64_8_8_1_30_6_months_validation_0_0_0_5_noise,
)

Observations

Using six months of data as a window to predict the stock price, and adding noise to the input training samples, the training loss and mean absolute error are decreasing gradually. In contrast, the validation loss and mean absolute error are fluctuating, i.e., both increasing and decreasing. The observed validation mean absolute error is slightly greater compared to that obtained using a high performance model with 3 LSTM layers and 3 Dense layers, also using six months of data as a window. One option to potentially decrease the validation mean absolute error further is to try different data augmentation techniques

• 3 LSTM(128-64-64) 3 Dense(8-8-1) - 6 months - noise(0.005)
• Epochs: 30
• Batch Size: 16
• Training Loss: 2.9440e-05
• Validation Loss: 7.4957e-05
• Training MAE: 0.0040
• Validation MAE: 0.0072

Jittering - 0.01

def add_noise(data, noise_level):
    # Generate random Gaussian noise. This noise has a mean of 0 and a standard deviation provided by 'noise_level'.
    # The size of the noise array is the same as the size of the input data.
    noise = np.random.normal(loc=0.0, scale=noise_level, size=data.shape)

    # Add the generated noise to the original data. This simulates real-world unpredictability in the data.
    return data + noise

# Let's add random noise to six-month training data window
x_train_6_months_noisy_0_0_1 = add_noise(x_train_6_months, noise_level=0.01)

Model Building with 3 LSTM and 3 Dense layers (128-64-64-8-8-1) and 30 epochs - 6 months

# Let's create a keras sequential model
model_128_64_64_8_8_1_30_6_months_validation_0_0_1_noise = Sequential()

# Let's add two LSTM layer's to the model network
model_128_64_64_8_8_1_30_6_months_validation_0_0_1_noise.add(
    LSTM(
        128,
        return_sequences=True,
        input_shape=(x_train_6_months_noisy_0_0_1.shape[1], 1),
    )
)
model_128_64_64_8_8_1_30_6_months_validation_0_0_1_noise.add(
    LSTM(64, return_sequences=True)
)
model_128_64_64_8_8_1_30_6_months_validation_0_0_1_noise.add(
    LSTM(64, return_sequences=False)
)

# Let's add dense layer to the model network
model_128_64_64_8_8_1_30_6_months_validation_0_0_1_noise.add(Dense(8))
model_128_64_64_8_8_1_30_6_months_validation_0_0_1_noise.add(Dense(8))
model_128_64_64_8_8_1_30_6_months_validation_0_0_1_noise.add(Dense(1))

# Let's build the model using mean_squared_error as the loss function
# and mean_absolute_error as the evaluation metric during the compilation process
model_128_64_64_8_8_1_30_6_months_validation_0_0_1_noise.compile(
    optimizer="adam", loss="mean_squared_error", metrics=["mean_absolute_error"]
)

# Let's define the callbacks for Model saving and Early stopping
cb_check = keras.callbacks.ModelCheckpoint(
    filepath="model_128_64_64_8_8_1_30_6_months_validation_0_0_1_noise_checkpoint_filepath",
    save_best_only=True,
    monitor="val_loss",
)

cb_early = keras.callbacks.EarlyStopping(monitor="val_loss", patience=30)

# Let's fit the sequential model with input features and output label
history_128_64_64_8_8_1_30_6_months_validation_0_0_1_noise = (
    model_128_64_64_8_8_1_30_6_months_validation_0_0_1_noise.fit(
        x_train_6_months_noisy_0_0_1,
        y_train_6_months,
        batch_size=16,
        epochs=30,
        callbacks=[cb_check, cb_early],
        validation_data=(x_valid_6_months, y_valid_6_months),
    )
)

Epoch 1/30
140/140 [==============================] - 25s 142ms/step - loss: 2.0123e-04 - mean_absolute_error: 0.0087 - val_loss: 4.8198e-04 - val_mean_absolute_error: 0.0171
Epoch 2/30
140/140 [==============================] - 19s 137ms/step - loss: 6.1152e-05 - mean_absolute_error: 0.0054 - val_loss: 3.0552e-04 - val_mean_absolute_error: 0.0147
Epoch 3/30
140/140 [==============================] - 19s 137ms/step - loss: 5.9178e-05 - mean_absolute_error: 0.0055 - val_loss: 2.6424e-04 - val_mean_absolute_error: 0.0137
Epoch 4/30
140/140 [==============================] - 19s 137ms/step - loss: 6.1496e-05 - mean_absolute_error: 0.0056 - val_loss: 2.5072e-04 - val_mean_absolute_error: 0.0133
Epoch 5/30
140/140 [==============================] - 17s 117ms/step - loss: 6.2748e-05 - mean_absolute_error: 0.0056 - val_loss: 3.1506e-04 - val_mean_absolute_error: 0.0137
Epoch 6/30
140/140 [==============================] - 19s 136ms/step - loss: 5.7641e-05 - mean_absolute_error: 0.0055 - val_loss: 1.9034e-04 - val_mean_absolute_error: 0.0115
Epoch 7/30
140/140 [==============================] - 21s 148ms/step - loss: 3.9105e-05 - mean_absolute_error: 0.0044 - val_loss: 1.7434e-04 - val_mean_absolute_error: 0.0110
Epoch 8/30
140/140 [==============================] - 19s 135ms/step - loss: 3.9338e-05 - mean_absolute_error: 0.0045 - val_loss: 1.7085e-04 - val_mean_absolute_error: 0.0109
Epoch 9/30
140/140 [==============================] - 16s 116ms/step - loss: 4.2850e-05 - mean_absolute_error: 0.0047 - val_loss: 1.8385e-04 - val_mean_absolute_error: 0.0108
Epoch 10/30
140/140 [==============================] - 29s 205ms/step - loss: 5.2392e-05 - mean_absolute_error: 0.0054 - val_loss: 1.5748e-04 - val_mean_absolute_error: 0.0103
Epoch 11/30
140/140 [==============================] - 16s 115ms/step - loss: 4.7295e-05 - mean_absolute_error: 0.0050 - val_loss: 2.6379e-04 - val_mean_absolute_error: 0.0133
Epoch 12/30
140/140 [==============================] - 18s 132ms/step - loss: 3.7962e-05 - mean_absolute_error: 0.0044 - val_loss: 1.5387e-04 - val_mean_absolute_error: 0.0103
Epoch 13/30
140/140 [==============================] - 16s 115ms/step - loss: 6.9921e-05 - mean_absolute_error: 0.0063 - val_loss: 1.7096e-04 - val_mean_absolute_error: 0.0109
Epoch 14/30
140/140 [==============================] - 16s 115ms/step - loss: 3.2735e-05 - mean_absolute_error: 0.0040 - val_loss: 1.7916e-04 - val_mean_absolute_error: 0.0107
Epoch 15/30
140/140 [==============================] - 16s 114ms/step - loss: 3.8989e-05 - mean_absolute_error: 0.0045 - val_loss: 1.7608e-04 - val_mean_absolute_error: 0.0105
Epoch 16/30
140/140 [==============================] - 16s 114ms/step - loss: 3.6161e-05 - mean_absolute_error: 0.0045 - val_loss: 1.7137e-04 - val_mean_absolute_error: 0.0105
Epoch 17/30
140/140 [==============================] - 19s 135ms/step - loss: 3.9404e-05 - mean_absolute_error: 0.0046 - val_loss: 1.4725e-04 - val_mean_absolute_error: 0.0101
Epoch 18/30
140/140 [==============================] - 21s 148ms/step - loss: 3.1460e-05 - mean_absolute_error: 0.0040 - val_loss: 1.3075e-04 - val_mean_absolute_error: 0.0094
Epoch 19/30
140/140 [==============================] - 17s 125ms/step - loss: 3.4776e-05 - mean_absolute_error: 0.0043 - val_loss: 1.3954e-04 - val_mean_absolute_error: 0.0097
Epoch 20/30
140/140 [==============================] - 20s 144ms/step - loss: 2.9954e-05 - mean_absolute_error: 0.0040 - val_loss: 1.2570e-04 - val_mean_absolute_error: 0.0093
Epoch 21/30
140/140 [==============================] - 17s 121ms/step - loss: 4.0631e-05 - mean_absolute_error: 0.0046 - val_loss: 1.4089e-04 - val_mean_absolute_error: 0.0097
Epoch 22/30
140/140 [==============================] - 17s 119ms/step - loss: 4.1711e-05 - mean_absolute_error: 0.0046 - val_loss: 1.2862e-04 - val_mean_absolute_error: 0.0093
Epoch 23/30
140/140 [==============================] - 17s 120ms/step - loss: 3.8155e-05 - mean_absolute_error: 0.0045 - val_loss: 1.3856e-04 - val_mean_absolute_error: 0.0097
Epoch 24/30
140/140 [==============================] - 20s 144ms/step - loss: 4.2444e-05 - mean_absolute_error: 0.0049 - val_loss: 1.1898e-04 - val_mean_absolute_error: 0.0090
Epoch 25/30
140/140 [==============================] - 17s 119ms/step - loss: 3.1332e-05 - mean_absolute_error: 0.0041 - val_loss: 1.3561e-04 - val_mean_absolute_error: 0.0096
Epoch 26/30
140/140 [==============================] - 25s 178ms/step - loss: 3.7604e-05 - mean_absolute_error: 0.0044 - val_loss: 1.3578e-04 - val_mean_absolute_error: 0.0097
Epoch 27/30
140/140 [==============================] - 20s 139ms/step - loss: 2.9834e-05 - mean_absolute_error: 0.0040 - val_loss: 1.0157e-04 - val_mean_absolute_error: 0.0083
Epoch 28/30
140/140 [==============================] - 17s 118ms/step - loss: 3.4429e-05 - mean_absolute_error: 0.0042 - val_loss: 1.0211e-04 - val_mean_absolute_error: 0.0083
Epoch 29/30
140/140 [==============================] - 16s 117ms/step - loss: 3.0734e-05 - mean_absolute_error: 0.0039 - val_loss: 1.3741e-04 - val_mean_absolute_error: 0.0097
Epoch 30/30
140/140 [==============================] - 16s 116ms/step - loss: 5.1233e-05 - mean_absolute_error: 0.0051 - val_loss: 1.2134e-04 - val_mean_absolute_error: 0.0091

# Let's plot the mean_squared_error and mean_absolute_error using model history on the entire dataset
plot_loss_and_mae_validation(
    model_name="model_128_64_64_8_8_1_30_6_months_validation_0_0_1_noise",
    model_history=history_128_64_64_8_8_1_30_6_months_validation_0_0_1_noise,
)

Observations

Using six months of data as a window to predict the stock price, and adding noise to the input training samples, the training loss and mean absolute error are decreasing gradually. In contrast, the validation loss and mean absolute error are fluctuating, i.e., both increasing and decreasing. The observed validation mean absolute error is greater than that obtained using a high-performance model with 3 LSTM layers and 3 Dense layers. It is also greater than when using noise level of 0.005 with the same six-month data window. One option to potentially decrease the validation mean absolute error further is to try different data augmentation techniques.

• 3 LSTM(128-64-64) 3 Dense(8-8-1) - 6 months - noise(0.0.1)
• Epochs: 30
• Batch Size: 16
• Training Loss: 5.1233e-05
• Validation Loss: 1.2134e-04
• Training MAE: 0.0051
• Validation MAE: 0.0091

Jittering - 0.03

def add_noise(data, noise_level):
    # Generate random Gaussian noise. This noise has a mean of 0 and a standard deviation provided by 'noise_level'.
    # The size of the noise array is the same as the size of the input data.
    noise = np.random.normal(loc=0.0, scale=noise_level, size=data.shape)

    # Add the generated noise to the original data. This simulates real-world unpredictability in the data.
    return data + noise

# Let's add random noise to six-month training data window
x_train_6_months_noisy_0_0_3 = add_noise(x_train_6_months, noise_level=0.03)

Model Building with 3 LSTM and 3 Dense layers (128-64-64-8-8-1) and 30 epochs - 6 months

# Let's create a keras sequential model
model_128_64_64_8_8_1_30_6_months_validation_0_0_3_noise = Sequential()

# Let's add two LSTM layer's to the model network
model_128_64_64_8_8_1_30_6_months_validation_0_0_3_noise.add(
    LSTM(
        128,
        return_sequences=True,
        input_shape=(x_train_6_months_noisy_0_0_3.shape[1], 1),
    )
)
model_128_64_64_8_8_1_30_6_months_validation_0_0_3_noise.add(
    LSTM(64, return_sequences=True)
)
model_128_64_64_8_8_1_30_6_months_validation_0_0_3_noise.add(
    LSTM(64, return_sequences=False)
)

# Let's add dense layer to the model network
model_128_64_64_8_8_1_30_6_months_validation_0_0_3_noise.add(Dense(8))
model_128_64_64_8_8_1_30_6_months_validation_0_0_3_noise.add(Dense(8))
model_128_64_64_8_8_1_30_6_months_validation_0_0_3_noise.add(Dense(1))

# Let's build the model using mean_squared_error as the loss function
# and mean_absolute_error as the evaluation metric during the compilation process
model_128_64_64_8_8_1_30_6_months_validation_0_0_3_noise.compile(
    optimizer="adam", loss="mean_squared_error", metrics=["mean_absolute_error"]
)

# Let's define the callbacks for Model saving and Early stopping
cb_check = keras.callbacks.ModelCheckpoint(
    filepath="model_128_64_64_8_8_1_30_6_months_validation_0_0_3_noise_checkpoint_filepath",
    save_best_only=True,
    monitor="val_loss",
)

cb_early = keras.callbacks.EarlyStopping(monitor="val_loss", patience=30)

# Let's fit the sequential model with input features and output label
history_128_64_64_8_8_1_30_6_months_validation_0_0_3_noise = (
    model_128_64_64_8_8_1_30_6_months_validation_0_0_3_noise.fit(
        x_train_6_months_noisy_0_0_3,
        y_train_6_months,
        batch_size=16,
        epochs=30,
        callbacks=[cb_check, cb_early],
        validation_data=(x_valid_6_months, y_valid_6_months),
    )
)

Epoch 1/30
140/140 [==============================] - 25s 143ms/step - loss: 3.5745e-04 - mean_absolute_error: 0.0105 - val_loss: 5.5851e-04 - val_mean_absolute_error: 0.0204
Epoch 2/30
140/140 [==============================] - 20s 140ms/step - loss: 1.3934e-04 - mean_absolute_error: 0.0089 - val_loss: 4.9627e-04 - val_mean_absolute_error: 0.0191
Epoch 3/30
140/140 [==============================] - 16s 117ms/step - loss: 1.3909e-04 - mean_absolute_error: 0.0083 - val_loss: 5.4799e-04 - val_mean_absolute_error: 0.0184
Epoch 4/30
140/140 [==============================] - 47s 335ms/step - loss: 1.4237e-04 - mean_absolute_error: 0.0086 - val_loss: 4.7874e-04 - val_mean_absolute_error: 0.0186
Epoch 5/30
140/140 [==============================] - 21s 151ms/step - loss: 1.0223e-04 - mean_absolute_error: 0.0075 - val_loss: 4.6309e-04 - val_mean_absolute_error: 0.0173
Epoch 6/30
140/140 [==============================] - 19s 138ms/step - loss: 9.5048e-05 - mean_absolute_error: 0.0072 - val_loss: 5.8912e-04 - val_mean_absolute_error: 0.0185
Epoch 7/30
140/140 [==============================] - 29s 207ms/step - loss: 1.0739e-04 - mean_absolute_error: 0.0077 - val_loss: 3.9224e-04 - val_mean_absolute_error: 0.0171
Epoch 8/30
140/140 [==============================] - 18s 124ms/step - loss: 9.2226e-05 - mean_absolute_error: 0.0071 - val_loss: 5.5259e-04 - val_mean_absolute_error: 0.0182
Epoch 9/30
140/140 [==============================] - 17s 124ms/step - loss: 9.8208e-05 - mean_absolute_error: 0.0075 - val_loss: 4.4100e-04 - val_mean_absolute_error: 0.0167
Epoch 10/30
140/140 [==============================] - 23s 164ms/step - loss: 1.0120e-04 - mean_absolute_error: 0.0073 - val_loss: 3.7670e-04 - val_mean_absolute_error: 0.0168
Epoch 11/30
140/140 [==============================] - 19s 132ms/step - loss: 9.0664e-05 - mean_absolute_error: 0.0071 - val_loss: 4.4445e-04 - val_mean_absolute_error: 0.0180
Epoch 12/30
140/140 [==============================] - 24s 170ms/step - loss: 9.3850e-05 - mean_absolute_error: 0.0072 - val_loss: 3.6712e-04 - val_mean_absolute_error: 0.0166
Epoch 13/30
140/140 [==============================] - 18s 130ms/step - loss: 9.4667e-05 - mean_absolute_error: 0.0071 - val_loss: 4.2668e-04 - val_mean_absolute_error: 0.0163
Epoch 14/30
140/140 [==============================] - 17s 123ms/step - loss: 1.1042e-04 - mean_absolute_error: 0.0078 - val_loss: 3.7163e-04 - val_mean_absolute_error: 0.0163
Epoch 15/30
140/140 [==============================] - 17s 121ms/step - loss: 8.4749e-05 - mean_absolute_error: 0.0069 - val_loss: 3.7594e-04 - val_mean_absolute_error: 0.0167
Epoch 16/30
140/140 [==============================] - 21s 153ms/step - loss: 8.9289e-05 - mean_absolute_error: 0.0070 - val_loss: 3.5129e-04 - val_mean_absolute_error: 0.0163
Epoch 17/30
140/140 [==============================] - 16s 116ms/step - loss: 8.8881e-05 - mean_absolute_error: 0.0070 - val_loss: 6.2682e-04 - val_mean_absolute_error: 0.0204
Epoch 18/30
140/140 [==============================] - 19s 139ms/step - loss: 8.2027e-05 - mean_absolute_error: 0.0066 - val_loss: 3.8040e-04 - val_mean_absolute_error: 0.0168
Epoch 19/30
140/140 [==============================] - 22s 156ms/step - loss: 1.3024e-04 - mean_absolute_error: 0.0078 - val_loss: 2.7619e-04 - val_mean_absolute_error: 0.0141
Epoch 20/30
140/140 [==============================] - 18s 124ms/step - loss: 1.0132e-04 - mean_absolute_error: 0.0075 - val_loss: 2.9942e-04 - val_mean_absolute_error: 0.0144
Epoch 21/30
140/140 [==============================] - 17s 123ms/step - loss: 8.4194e-05 - mean_absolute_error: 0.0069 - val_loss: 3.6085e-04 - val_mean_absolute_error: 0.0155
Epoch 22/30
140/140 [==============================] - 23s 165ms/step - loss: 7.2242e-05 - mean_absolute_error: 0.0064 - val_loss: 2.7508e-04 - val_mean_absolute_error: 0.0143
Epoch 23/30
140/140 [==============================] - 19s 132ms/step - loss: 8.5438e-05 - mean_absolute_error: 0.0069 - val_loss: 2.9169e-04 - val_mean_absolute_error: 0.0140
Epoch 24/30
140/140 [==============================] - 17s 122ms/step - loss: 7.3296e-05 - mean_absolute_error: 0.0064 - val_loss: 3.0016e-04 - val_mean_absolute_error: 0.0149
Epoch 25/30
140/140 [==============================] - 17s 122ms/step - loss: 8.0779e-05 - mean_absolute_error: 0.0066 - val_loss: 2.9484e-04 - val_mean_absolute_error: 0.0149
Epoch 26/30
140/140 [==============================] - 19s 132ms/step - loss: 9.6908e-05 - mean_absolute_error: 0.0072 - val_loss: 3.0728e-04 - val_mean_absolute_error: 0.0154
Epoch 27/30
140/140 [==============================] - 20s 140ms/step - loss: 6.5840e-05 - mean_absolute_error: 0.0061 - val_loss: 4.9312e-04 - val_mean_absolute_error: 0.0184
Epoch 28/30
140/140 [==============================] - 22s 156ms/step - loss: 7.2747e-05 - mean_absolute_error: 0.0064 - val_loss: 2.3099e-04 - val_mean_absolute_error: 0.0135
Epoch 29/30
140/140 [==============================] - 18s 127ms/step - loss: 7.4100e-05 - mean_absolute_error: 0.0063 - val_loss: 2.8835e-04 - val_mean_absolute_error: 0.0143
Epoch 30/30
140/140 [==============================] - 18s 131ms/step - loss: 6.8357e-05 - mean_absolute_error: 0.0061 - val_loss: 2.4819e-04 - val_mean_absolute_error: 0.0132

# Let's plot the mean_squared_error and mean_absolute_error using model history on the entire dataset
plot_loss_and_mae_validation(
    model_name="model_128_64_64_8_8_1_30_6_months_validation_0_0_3_noise",
    model_history=history_128_64_64_8_8_1_30_6_months_validation_0_0_3_noise,
)

Observations

Using six months of data as a window to predict stock prices, and adding noise(0.03) to the input training samples, we see that the training loss and mean absolute error are gradually decreasing. However, the validation loss and mean absolute error are fluctuating, i.e., both increasing and decreasing. The observed validation mean absolute error is greater than that obtained using a high-performance model with 3 LSTM layers and 3 Dense layers. It is also greater than when using noise levels of 0.005 and 0.01 with the same six-month data window. One option to potentially decrease the validation mean absolute error further is to try different data augmentation techniques.

• 3 LSTM(128-64-64) 3 Dense(8-8-1) - 6 months - noise(0.0.3)
• Epochs: 30
• Batch Size: 16
• Training Loss: 6.8357e-05
• Validation Loss: 2.4819e-04
• Training MAE: 0.0061
• Validation MAE: 0.0132

Scaling

def scaling(data, sigma):
    # Apply scaling augmentation to the dataset by multiplying with a random normal factor.
    factor = np.random.normal(loc=1.0, scale=sigma, size=(data.shape[0], data.shape[2]))
    return np.multiply(data, factor[:, np.newaxis, :])

sigma = 0.01

# Let's scale each time series of six-month training data window by constant amount
x_train_6_months_scaled_0_0_1 = scaling(x_train_6_months, sigma=0.01)

Model Building with 3 LSTM and 3 Dense layers (128-64-64-8-8-1) and 30 epochs - 6 months

# Let's create a keras sequential model
model_128_64_64_8_8_1_30_6_months_validation_0_0_1_scaled = Sequential()

# Let's add two LSTM layer's to the model network
model_128_64_64_8_8_1_30_6_months_validation_0_0_1_scaled.add(
    LSTM(
        128,
        return_sequences=True,
        input_shape=(x_train_6_months_scaled_0_0_1.shape[1], 1),
    )
)
model_128_64_64_8_8_1_30_6_months_validation_0_0_1_scaled.add(
    LSTM(64, return_sequences=True)
)
model_128_64_64_8_8_1_30_6_months_validation_0_0_1_scaled.add(
    LSTM(64, return_sequences=False)
)

# Let's add dense layer to the model network
model_128_64_64_8_8_1_30_6_months_validation_0_0_1_scaled.add(Dense(8))
model_128_64_64_8_8_1_30_6_months_validation_0_0_1_scaled.add(Dense(8))
model_128_64_64_8_8_1_30_6_months_validation_0_0_1_scaled.add(Dense(1))

# Let's build the model using mean_squared_error as the loss function
# and mean_absolute_error as the evaluation metric during the compilation process
model_128_64_64_8_8_1_30_6_months_validation_0_0_1_scaled.compile(
    optimizer="adam", loss="mean_squared_error", metrics=["mean_absolute_error"]
)

# Let's define the callbacks for Model saving and Early stopping
cb_check = keras.callbacks.ModelCheckpoint(
    filepath="model_128_64_64_8_8_1_30_6_months_validation_0_0_1_scaled_checkpoint_filepath",
    save_best_only=True,
    monitor="val_loss",
)

cb_early = keras.callbacks.EarlyStopping(monitor="val_loss", patience=30)

# Let's fit the sequential model with input features and output label
history_128_64_64_8_8_1_30_6_months_validation_0_0_1_scaled = (
    model_128_64_64_8_8_1_30_6_months_validation_0_0_1_scaled.fit(
        x_train_6_months_scaled_0_0_1,
        y_train_6_months,
        batch_size=16,
        epochs=30,
        callbacks=[cb_check, cb_early],
        validation_data=(x_valid_6_months, y_valid_6_months),
    )
)

Epoch 1/30
140/140 [==============================] - 48s 146ms/step - loss: 2.1112e-04 - mean_absolute_error: 0.0081 - val_loss: 3.9642e-04 - val_mean_absolute_error: 0.0168
Epoch 2/30
140/140 [==============================] - 17s 120ms/step - loss: 5.5765e-05 - mean_absolute_error: 0.0048 - val_loss: 8.5301e-04 - val_mean_absolute_error: 0.0235
Epoch 3/30
140/140 [==============================] - 24s 169ms/step - loss: 6.5409e-05 - mean_absolute_error: 0.0056 - val_loss: 2.7357e-04 - val_mean_absolute_error: 0.0137
Epoch 4/30
140/140 [==============================] - 25s 178ms/step - loss: 4.2506e-05 - mean_absolute_error: 0.0042 - val_loss: 2.2954e-04 - val_mean_absolute_error: 0.0123
Epoch 5/30
140/140 [==============================] - 22s 158ms/step - loss: 4.4466e-05 - mean_absolute_error: 0.0046 - val_loss: 2.0410e-04 - val_mean_absolute_error: 0.0114
Epoch 6/30
140/140 [==============================] - 18s 128ms/step - loss: 3.5767e-05 - mean_absolute_error: 0.0038 - val_loss: 4.9218e-04 - val_mean_absolute_error: 0.0187
Epoch 7/30
140/140 [==============================] - 21s 150ms/step - loss: 3.5594e-05 - mean_absolute_error: 0.0038 - val_loss: 1.8751e-04 - val_mean_absolute_error: 0.0114
Epoch 8/30
140/140 [==============================] - 24s 169ms/step - loss: 3.6762e-05 - mean_absolute_error: 0.0042 - val_loss: 1.2631e-04 - val_mean_absolute_error: 0.0092
Epoch 9/30
140/140 [==============================] - 21s 148ms/step - loss: 2.7547e-05 - mean_absolute_error: 0.0034 - val_loss: 1.4471e-04 - val_mean_absolute_error: 0.0097
Epoch 10/30
140/140 [==============================] - 24s 169ms/step - loss: 3.1828e-05 - mean_absolute_error: 0.0037 - val_loss: 1.2572e-04 - val_mean_absolute_error: 0.0094
Epoch 11/30
140/140 [==============================] - 22s 160ms/step - loss: 2.9372e-05 - mean_absolute_error: 0.0036 - val_loss: 9.9894e-05 - val_mean_absolute_error: 0.0085
Epoch 12/30
140/140 [==============================] - 18s 131ms/step - loss: 2.3871e-05 - mean_absolute_error: 0.0034 - val_loss: 1.3626e-04 - val_mean_absolute_error: 0.0098
Epoch 13/30
140/140 [==============================] - 20s 141ms/step - loss: 2.2503e-05 - mean_absolute_error: 0.0031 - val_loss: 1.2725e-04 - val_mean_absolute_error: 0.0089
Epoch 14/30
140/140 [==============================] - 19s 134ms/step - loss: 2.8579e-05 - mean_absolute_error: 0.0035 - val_loss: 1.0558e-04 - val_mean_absolute_error: 0.0084
Epoch 15/30
140/140 [==============================] - 18s 130ms/step - loss: 2.8531e-05 - mean_absolute_error: 0.0035 - val_loss: 1.0412e-04 - val_mean_absolute_error: 0.0085
Epoch 16/30
140/140 [==============================] - 18s 129ms/step - loss: 2.1112e-05 - mean_absolute_error: 0.0032 - val_loss: 1.7780e-04 - val_mean_absolute_error: 0.0109
Epoch 17/30
140/140 [==============================] - 49s 353ms/step - loss: 1.9896e-05 - mean_absolute_error: 0.0030 - val_loss: 7.4091e-05 - val_mean_absolute_error: 0.0072
Epoch 18/30
140/140 [==============================] - 20s 130ms/step - loss: 2.0414e-05 - mean_absolute_error: 0.0032 - val_loss: 1.0079e-04 - val_mean_absolute_error: 0.0082
Epoch 19/30
140/140 [==============================] - 17s 123ms/step - loss: 1.7873e-05 - mean_absolute_error: 0.0028 - val_loss: 1.1201e-04 - val_mean_absolute_error: 0.0083
Epoch 20/30
140/140 [==============================] - 17s 122ms/step - loss: 2.4922e-05 - mean_absolute_error: 0.0035 - val_loss: 9.5491e-05 - val_mean_absolute_error: 0.0077
Epoch 21/30
140/140 [==============================] - 19s 133ms/step - loss: 2.2455e-05 - mean_absolute_error: 0.0033 - val_loss: 1.2721e-04 - val_mean_absolute_error: 0.0091
Epoch 22/30
140/140 [==============================] - 22s 154ms/step - loss: 2.1280e-05 - mean_absolute_error: 0.0032 - val_loss: 6.5617e-05 - val_mean_absolute_error: 0.0068
Epoch 23/30
140/140 [==============================] - 17s 122ms/step - loss: 1.8084e-05 - mean_absolute_error: 0.0029 - val_loss: 7.3076e-05 - val_mean_absolute_error: 0.0070
Epoch 24/30
140/140 [==============================] - 20s 145ms/step - loss: 1.5254e-05 - mean_absolute_error: 0.0026 - val_loss: 6.0017e-05 - val_mean_absolute_error: 0.0065
Epoch 25/30
140/140 [==============================] - 17s 123ms/step - loss: 2.6946e-05 - mean_absolute_error: 0.0036 - val_loss: 8.4527e-05 - val_mean_absolute_error: 0.0074
Epoch 26/30
140/140 [==============================] - 19s 134ms/step - loss: 2.4488e-05 - mean_absolute_error: 0.0034 - val_loss: 6.2392e-05 - val_mean_absolute_error: 0.0064
Epoch 27/30
140/140 [==============================] - 17s 122ms/step - loss: 1.6289e-05 - mean_absolute_error: 0.0026 - val_loss: 6.1783e-05 - val_mean_absolute_error: 0.0064
Epoch 28/30
140/140 [==============================] - 17s 121ms/step - loss: 1.5256e-05 - mean_absolute_error: 0.0026 - val_loss: 1.5944e-04 - val_mean_absolute_error: 0.0107
Epoch 29/30
140/140 [==============================] - 17s 122ms/step - loss: 1.2406e-05 - mean_absolute_error: 0.0023 - val_loss: 7.8200e-05 - val_mean_absolute_error: 0.0070
Epoch 30/30
140/140 [==============================] - 17s 122ms/step - loss: 1.1455e-05 - mean_absolute_error: 0.0022 - val_loss: 8.3085e-05 - val_mean_absolute_error: 0.0074

# Let's plot the mean_squared_error and mean_absolute_error using model history on the entire dataset
plot_loss_and_mae_validation(
    model_name="model_128_64_64_8_8_1_30_6_months_validation_0_0_1_scaled",
    model_history=history_128_64_64_8_8_1_30_6_months_validation_0_0_1_scaled,
)

Observations

Using six months of data as a window to predict the stock price, and scaling by constant amount to the input training samples, the training loss and mean absolute error are decreasing gradually. In contrast, the validation loss and mean absolute error are fluctuating, i.e., both increasing and decreasing. The observed validation mean absolute error is greater than that obtained using a high-performance model with 3 LSTM layers and 3 Dense layers. It is less than when using noise levels of 0.03 and 0.01 with the same six-month data window. One option to potentially decrease the validation mean absolute error further is to increase/decrease the scaling magnitude.

• 3 LSTM(128-64-64) 3 Dense(8-8-1) - 6 months - scaling(0.0.1)
• Epochs: 30
• Batch Size: 16
• Training Loss: 1.1455e-05
• Validation Loss: 8.3085e-05
• Training MAE: 0.0022
• Validation MAE: 0.0074

sigma = 0.1

# Let's scale each time series of six-month training data window by constant amount
x_train_6_months_scaled_0_1 = scaling(x_train_6_months, sigma=0.1)

Model Building with 3 LSTM and 3 Dense layers (128-64-64-8-8-1) and 30 epochs - 6 months

# Let's create a keras sequential model
model_128_64_64_8_8_1_30_6_months_validation_0_1_scaled = Sequential()

# Let's add two LSTM layer's to the model network
model_128_64_64_8_8_1_30_6_months_validation_0_1_scaled.add(
    LSTM(
        128,
        return_sequences=True,
        input_shape=(x_train_6_months_scaled_0_1.shape[1], 1),
    )
)
model_128_64_64_8_8_1_30_6_months_validation_0_1_scaled.add(
    LSTM(64, return_sequences=True)
)
model_128_64_64_8_8_1_30_6_months_validation_0_1_scaled.add(
    LSTM(64, return_sequences=False)
)

# Let's add dense layer to the model network
model_128_64_64_8_8_1_30_6_months_validation_0_1_scaled.add(Dense(8))
model_128_64_64_8_8_1_30_6_months_validation_0_1_scaled.add(Dense(8))
model_128_64_64_8_8_1_30_6_months_validation_0_1_scaled.add(Dense(1))

# Let's build the model using mean_squared_error as the loss function
# and mean_absolute_error as the evaluation metric during the compilation process
model_128_64_64_8_8_1_30_6_months_validation_0_1_scaled.compile(
    optimizer="adam", loss="mean_squared_error", metrics=["mean_absolute_error"]
)

# Let's define the callbacks for Model saving and Early stopping
cb_check = keras.callbacks.ModelCheckpoint(
    filepath="model_128_64_64_8_8_1_30_6_months_validation_0_1_scaled_checkpoint_filepath",
    save_best_only=True,
    monitor="val_loss",
)

cb_early = keras.callbacks.EarlyStopping(monitor="val_loss", patience=30)

# Let's fit the sequential model with input features and output label
history_128_64_64_8_8_1_30_6_months_validation_0_1_scaled = (
    model_128_64_64_8_8_1_30_6_months_validation_0_1_scaled.fit(
        x_train_6_months_scaled_0_1,
        y_train_6_months,
        batch_size=16,
        epochs=30,
        callbacks=[cb_check, cb_early],
        validation_data=(x_valid_6_months, y_valid_6_months),
    )
)

Epoch 1/30
140/140 [==============================] - 26s 161ms/step - loss: 4.0306e-04 - mean_absolute_error: 0.0113 - val_loss: 4.1517e-04 - val_mean_absolute_error: 0.0170
Epoch 2/30
140/140 [==============================] - 25s 176ms/step - loss: 1.1064e-04 - mean_absolute_error: 0.0063 - val_loss: 3.8104e-04 - val_mean_absolute_error: 0.0164
Epoch 3/30
140/140 [==============================] - 19s 132ms/step - loss: 1.5969e-04 - mean_absolute_error: 0.0079 - val_loss: 4.1548e-04 - val_mean_absolute_error: 0.0167
Epoch 4/30
140/140 [==============================] - 22s 156ms/step - loss: 1.0702e-04 - mean_absolute_error: 0.0066 - val_loss: 3.3518e-04 - val_mean_absolute_error: 0.0150
Epoch 5/30
140/140 [==============================] - 23s 166ms/step - loss: 1.3287e-04 - mean_absolute_error: 0.0075 - val_loss: 2.3892e-04 - val_mean_absolute_error: 0.0123
Epoch 6/30
140/140 [==============================] - 23s 163ms/step - loss: 1.1654e-04 - mean_absolute_error: 0.0071 - val_loss: 1.8928e-04 - val_mean_absolute_error: 0.0113
Epoch 7/30
140/140 [==============================] - 18s 127ms/step - loss: 1.0733e-04 - mean_absolute_error: 0.0066 - val_loss: 2.1597e-04 - val_mean_absolute_error: 0.0120
Epoch 8/30
140/140 [==============================] - 18s 125ms/step - loss: 1.1854e-04 - mean_absolute_error: 0.0072 - val_loss: 2.2720e-04 - val_mean_absolute_error: 0.0123
Epoch 9/30
140/140 [==============================] - 19s 133ms/step - loss: 9.7908e-05 - mean_absolute_error: 0.0065 - val_loss: 3.4033e-04 - val_mean_absolute_error: 0.0152
Epoch 10/30
140/140 [==============================] - 22s 154ms/step - loss: 9.3616e-05 - mean_absolute_error: 0.0061 - val_loss: 3.4203e-04 - val_mean_absolute_error: 0.0153
Epoch 11/30
140/140 [==============================] - 38s 271ms/step - loss: 8.7783e-05 - mean_absolute_error: 0.0059 - val_loss: 1.0655e-04 - val_mean_absolute_error: 0.0087
Epoch 12/30
140/140 [==============================] - 18s 126ms/step - loss: 9.7638e-05 - mean_absolute_error: 0.0063 - val_loss: 1.6793e-04 - val_mean_absolute_error: 0.0104
Epoch 13/30
140/140 [==============================] - 16s 117ms/step - loss: 1.0461e-04 - mean_absolute_error: 0.0069 - val_loss: 1.6288e-04 - val_mean_absolute_error: 0.0106
Epoch 14/30
140/140 [==============================] - 16s 116ms/step - loss: 8.6785e-05 - mean_absolute_error: 0.0058 - val_loss: 2.1689e-04 - val_mean_absolute_error: 0.0118
Epoch 15/30
140/140 [==============================] - 16s 115ms/step - loss: 8.2228e-05 - mean_absolute_error: 0.0055 - val_loss: 1.2259e-04 - val_mean_absolute_error: 0.0090
Epoch 16/30
140/140 [==============================] - 17s 123ms/step - loss: 9.7917e-05 - mean_absolute_error: 0.0064 - val_loss: 1.1400e-04 - val_mean_absolute_error: 0.0089
Epoch 17/30
140/140 [==============================] - 18s 127ms/step - loss: 9.2426e-05 - mean_absolute_error: 0.0061 - val_loss: 1.1188e-04 - val_mean_absolute_error: 0.0088
Epoch 18/30
140/140 [==============================] - 26s 183ms/step - loss: 8.1120e-05 - mean_absolute_error: 0.0053 - val_loss: 1.0042e-04 - val_mean_absolute_error: 0.0084
Epoch 19/30
140/140 [==============================] - 19s 137ms/step - loss: 8.4124e-05 - mean_absolute_error: 0.0059 - val_loss: 3.3559e-04 - val_mean_absolute_error: 0.0154
Epoch 20/30
140/140 [==============================] - 41s 292ms/step - loss: 1.0398e-04 - mean_absolute_error: 0.0064 - val_loss: 8.3806e-05 - val_mean_absolute_error: 0.0078
Epoch 21/30
140/140 [==============================] - 29s 203ms/step - loss: 8.2341e-05 - mean_absolute_error: 0.0056 - val_loss: 1.1216e-04 - val_mean_absolute_error: 0.0085
Epoch 22/30
140/140 [==============================] - 27s 195ms/step - loss: 8.9470e-05 - mean_absolute_error: 0.0061 - val_loss: 8.2286e-05 - val_mean_absolute_error: 0.0077
Epoch 23/30
140/140 [==============================] - 21s 151ms/step - loss: 9.2270e-05 - mean_absolute_error: 0.0061 - val_loss: 7.8881e-05 - val_mean_absolute_error: 0.0074
Epoch 24/30
140/140 [==============================] - 17s 124ms/step - loss: 8.2585e-05 - mean_absolute_error: 0.0059 - val_loss: 1.9544e-04 - val_mean_absolute_error: 0.0113
Epoch 25/30
140/140 [==============================] - 17s 125ms/step - loss: 1.0515e-04 - mean_absolute_error: 0.0069 - val_loss: 4.0517e-04 - val_mean_absolute_error: 0.0177
Epoch 26/30
140/140 [==============================] - 22s 159ms/step - loss: 8.8145e-05 - mean_absolute_error: 0.0062 - val_loss: 6.8367e-05 - val_mean_absolute_error: 0.0071
Epoch 27/30
140/140 [==============================] - 24s 167ms/step - loss: 7.2780e-05 - mean_absolute_error: 0.0053 - val_loss: 1.0030e-04 - val_mean_absolute_error: 0.0079
Epoch 28/30
140/140 [==============================] - 21s 150ms/step - loss: 7.5696e-05 - mean_absolute_error: 0.0056 - val_loss: 1.4642e-04 - val_mean_absolute_error: 0.0098
Epoch 29/30
140/140 [==============================] - 30s 213ms/step - loss: 8.2742e-05 - mean_absolute_error: 0.0059 - val_loss: 7.4825e-05 - val_mean_absolute_error: 0.0073
Epoch 30/30
140/140 [==============================] - 28s 197ms/step - loss: 8.9622e-05 - mean_absolute_error: 0.0058 - val_loss: 6.3728e-05 - val_mean_absolute_error: 0.0066

# Let's plot the mean_squared_error and mean_absolute_error using model history on the entire dataset
plot_loss_and_mae_validation(
    model_name="model_128_64_64_8_8_1_30_6_months_validation_0_1_scaled",
    model_history=history_128_64_64_8_8_1_30_6_months_validation_0_1_scaled,
)

Observations

Using six months of data as a window to predict the stock price, and scaling by constant amount to the input training samples, the training loss and mean absolute error are decreasing gradually. In contrast, the validation loss and mean absolute error are fluctuating, i.e., both increasing and decreasing. The observed validation mean absolute error is similar to that obtained using a high-performance model with 3 LSTM layers and 3 Dense layers. It is less than when using noise levels of 0.0053, 0.03 and 0.01 with the same six-month data window. One option to potentially decrease the validation mean absolute error further is to increase/decrease the scaling magnitude.

• 3 LSTM(128-64-64) 3 Dense(8-8-1) - 6 months - scaling(0.1)
• Epochs: 30
• Batch Size: 16
• Training Loss: 8.9622e-05
• Validation Loss: 6.3728e-05
• Training MAE: 0.0058
• Validation MAE: 0.0066

sigma = 0.3

# Let's scale each time series of six-month training data window by constant amount
x_train_6_months_scaled_0_3 = scaling(x_train_6_months, sigma=0.3)

Model Building with 3 LSTM and 3 Dense layers (128-64-64-8-8-1) and 30 epochs - 6 months

# Let's create a keras sequential model
model_128_64_64_8_8_1_30_6_months_validation_0_3_scaled = Sequential()

# Let's add two LSTM layer's to the model network
model_128_64_64_8_8_1_30_6_months_validation_0_3_scaled.add(
    LSTM(
        128,
        return_sequences=True,
        input_shape=(x_train_6_months_scaled_0_3.shape[1], 1),
    )
)
model_128_64_64_8_8_1_30_6_months_validation_0_3_scaled.add(
    LSTM(64, return_sequences=True)
)
model_128_64_64_8_8_1_30_6_months_validation_0_3_scaled.add(
    LSTM(64, return_sequences=False)
)

# Let's add dense layer to the model network
model_128_64_64_8_8_1_30_6_months_validation_0_3_scaled.add(Dense(8))
model_128_64_64_8_8_1_30_6_months_validation_0_3_scaled.add(Dense(8))
model_128_64_64_8_8_1_30_6_months_validation_0_3_scaled.add(Dense(1))

# Let's build the model using mean_squared_error as the loss function
# and mean_absolute_error as the evaluation metric during the compilation process
model_128_64_64_8_8_1_30_6_months_validation_0_3_scaled.compile(
    optimizer="adam", loss="mean_squared_error", metrics=["mean_absolute_error"]
)

# Let's define the callbacks for Model saving and Early stopping
cb_check = keras.callbacks.ModelCheckpoint(
    filepath="model_128_64_64_8_8_1_30_6_months_validation_0_3_scaled_checkpoint_filepath",
    save_best_only=True,
    monitor="val_loss",
)

cb_early = keras.callbacks.EarlyStopping(monitor="val_loss", patience=30)

# Let's fit the sequential model with input features and output label
history_128_64_64_8_8_1_30_6_months_validation_0_3_scaled = (
    model_128_64_64_8_8_1_30_6_months_validation_0_3_scaled.fit(
        x_train_6_months_scaled_0_3,
        y_train_6_months,
        batch_size=16,
        epochs=30,
        callbacks=[cb_check, cb_early],
        validation_data=(x_valid_6_months, y_valid_6_months),
    )
)

Epoch 1/30
140/140 [==============================] - 27s 156ms/step - loss: 7.5104e-04 - mean_absolute_error: 0.0172 - val_loss: 4.4250e-04 - val_mean_absolute_error: 0.0167
Epoch 2/30
140/140 [==============================] - 44s 315ms/step - loss: 5.8796e-04 - mean_absolute_error: 0.0150 - val_loss: 4.3974e-04 - val_mean_absolute_error: 0.0168
Epoch 3/30
140/140 [==============================] - 21s 148ms/step - loss: 5.7471e-04 - mean_absolute_error: 0.0153 - val_loss: 4.0774e-04 - val_mean_absolute_error: 0.0161
Epoch 4/30
140/140 [==============================] - 32s 228ms/step - loss: 5.3428e-04 - mean_absolute_error: 0.0143 - val_loss: 2.6578e-04 - val_mean_absolute_error: 0.0136
Epoch 5/30
140/140 [==============================] - 54s 375ms/step - loss: 5.6877e-04 - mean_absolute_error: 0.0149 - val_loss: 2.5775e-04 - val_mean_absolute_error: 0.0132
Epoch 6/30
140/140 [==============================] - 19s 133ms/step - loss: 5.4194e-04 - mean_absolute_error: 0.0144 - val_loss: 2.6982e-04 - val_mean_absolute_error: 0.0135
Epoch 7/30
140/140 [==============================] - 21s 151ms/step - loss: 5.1300e-04 - mean_absolute_error: 0.0139 - val_loss: 2.4371e-04 - val_mean_absolute_error: 0.0128
Epoch 8/30
140/140 [==============================] - 19s 137ms/step - loss: 5.8509e-04 - mean_absolute_error: 0.0153 - val_loss: 7.9469e-04 - val_mean_absolute_error: 0.0244
Epoch 9/30
140/140 [==============================] - 18s 128ms/step - loss: 5.0479e-04 - mean_absolute_error: 0.0140 - val_loss: 5.8605e-04 - val_mean_absolute_error: 0.0197
Epoch 10/30
140/140 [==============================] - 16s 114ms/step - loss: 5.3192e-04 - mean_absolute_error: 0.0139 - val_loss: 2.9518e-04 - val_mean_absolute_error: 0.0137
Epoch 11/30
140/140 [==============================] - 17s 121ms/step - loss: 5.1206e-04 - mean_absolute_error: 0.0138 - val_loss: 3.9626e-04 - val_mean_absolute_error: 0.0154
Epoch 12/30
140/140 [==============================] - 16s 112ms/step - loss: 5.6662e-04 - mean_absolute_error: 0.0149 - val_loss: 4.2500e-04 - val_mean_absolute_error: 0.0165
Epoch 13/30
140/140 [==============================] - 16s 114ms/step - loss: 4.9844e-04 - mean_absolute_error: 0.0133 - val_loss: 5.5123e-04 - val_mean_absolute_error: 0.0192
Epoch 14/30
140/140 [==============================] - 16s 112ms/step - loss: 5.0899e-04 - mean_absolute_error: 0.0141 - val_loss: 3.6910e-04 - val_mean_absolute_error: 0.0150
Epoch 15/30
140/140 [==============================] - 18s 132ms/step - loss: 4.9237e-04 - mean_absolute_error: 0.0133 - val_loss: 3.9160e-04 - val_mean_absolute_error: 0.0159
Epoch 16/30
140/140 [==============================] - 18s 126ms/step - loss: 5.0104e-04 - mean_absolute_error: 0.0138 - val_loss: 5.7740e-04 - val_mean_absolute_error: 0.0202
Epoch 17/30
140/140 [==============================] - 22s 155ms/step - loss: 4.9439e-04 - mean_absolute_error: 0.0139 - val_loss: 1.2996e-04 - val_mean_absolute_error: 0.0094
Epoch 18/30
140/140 [==============================] - 16s 116ms/step - loss: 4.8631e-04 - mean_absolute_error: 0.0135 - val_loss: 6.8298e-04 - val_mean_absolute_error: 0.0241
Epoch 19/30
140/140 [==============================] - 18s 132ms/step - loss: 5.3243e-04 - mean_absolute_error: 0.0148 - val_loss: 8.4516e-04 - val_mean_absolute_error: 0.0259
Epoch 20/30
140/140 [==============================] - 22s 153ms/step - loss: 4.9320e-04 - mean_absolute_error: 0.0136 - val_loss: 1.1398e-04 - val_mean_absolute_error: 0.0089
Epoch 21/30
140/140 [==============================] - 17s 117ms/step - loss: 7.5468e-04 - mean_absolute_error: 0.0164 - val_loss: 2.8580e-04 - val_mean_absolute_error: 0.0132
Epoch 22/30
140/140 [==============================] - 16s 114ms/step - loss: 4.9806e-04 - mean_absolute_error: 0.0136 - val_loss: 1.8046e-04 - val_mean_absolute_error: 0.0107
Epoch 23/30
140/140 [==============================] - 16s 115ms/step - loss: 4.9495e-04 - mean_absolute_error: 0.0133 - val_loss: 1.8484e-04 - val_mean_absolute_error: 0.0107
Epoch 24/30
140/140 [==============================] - 16s 113ms/step - loss: 4.7500e-04 - mean_absolute_error: 0.0130 - val_loss: 1.8972e-04 - val_mean_absolute_error: 0.0116
Epoch 25/30
140/140 [==============================] - 16s 116ms/step - loss: 4.8901e-04 - mean_absolute_error: 0.0136 - val_loss: 1.3667e-04 - val_mean_absolute_error: 0.0097
Epoch 26/30
140/140 [==============================] - 17s 125ms/step - loss: 4.8336e-04 - mean_absolute_error: 0.0132 - val_loss: 1.3636e-04 - val_mean_absolute_error: 0.0096
Epoch 27/30
140/140 [==============================] - 16s 113ms/step - loss: 4.7729e-04 - mean_absolute_error: 0.0131 - val_loss: 3.7270e-04 - val_mean_absolute_error: 0.0153
Epoch 28/30
140/140 [==============================] - 16s 114ms/step - loss: 4.8126e-04 - mean_absolute_error: 0.0132 - val_loss: 1.1515e-04 - val_mean_absolute_error: 0.0087
Epoch 29/30
140/140 [==============================] - 16s 113ms/step - loss: 4.8890e-04 - mean_absolute_error: 0.0132 - val_loss: 1.6291e-04 - val_mean_absolute_error: 0.0105
Epoch 30/30
140/140 [==============================] - 21s 149ms/step - loss: 5.1943e-04 - mean_absolute_error: 0.0143 - val_loss: 1.0191e-04 - val_mean_absolute_error: 0.0083

# Let's plot the mean_squared_error and mean_absolute_error using model history on the entire dataset
plot_loss_and_mae_validation(
    model_name="model_128_64_64_8_8_1_30_6_months_validation_0_3_scaled",
    model_history=history_128_64_64_8_8_1_30_6_months_validation_0_3_scaled,
)

Observations

Using six months of data as a window to predict the stock price, and scaling by constant amount(0.3) to the input training samples, the training loss and mean absolute error are decreasing gradually. In contrast, the validation loss and mean absolute error are fluctuating, i.e., both increasing and decreasing. The observed validation mean absolute error is greater than to that obtained using a high-performance model with 3 LSTM layers and 3 Dense layers. It is less than when using noise levels of 0.03 and 0.01 with the same six-month data window. One option to potentially decrease the validation mean absolute error further is to increase/decrease the scaling magnitude.

• 3 LSTM(128-64-64) 3 Dense(8-8-1) - 6 months - 0.3 Scaling
• Epochs: 30
• Batch Size: 16
• Training Loss: 5.1943e-04
• Validation Loss: 1.0191e-04
• Training MAE: 0.0143
• Validation MAE: 0.0083

Magnitude Warping

from scipy.interpolate import CubicSpline


def magnitude_warp(x, sigma, knot):
    # Import the CubicSpline function from scipy for smooth interpolation
    from scipy.interpolate import CubicSpline

    # Create an array of indices representing each time step in the data
    orig_steps = np.arange(x.shape[1])

    # Generate random factors for warping, normally distributed around 1 with a deviation of sigma
    random_warps = np.random.normal(
        loc=1.0, scale=sigma, size=(x.shape[0], knot + 2, x.shape[2])
    )
    # Define the time steps at which to apply these random factors
    warp_steps = (
        np.ones((x.shape[2], 1)) * (np.linspace(0, x.shape[1] - 1.0, num=knot + 2))
    ).T
    # Initialize an output array with the same shape as the input, filled with zeros
    ret = np.zeros_like(x)
    # Loop over each sample in the dataset
    for i, pat in enumerate(x):
        # Compute the warping function for each feature using cubic spline interpolation
        warper = np.array(
            [
                CubicSpline(warp_steps[:, dim], random_warps[i, :, dim])(orig_steps)
                for dim in range(x.shape[2])
            ]
        ).T
        # Apply the warping function to the data by element-wise multiplication
        ret[i] = pat * warper

    # Return the warped dataset
    return ret

sigma = 0.2, knot = 1

# Let's warp each time series of six-month training data window by constant amount
x_train_6_months_mag_warped_0_2_1 = magnitude_warp(x_train_6_months, sigma=0.2, knot=1)

Model Building with 3 LSTM and 3 Dense layers (128-64-64-8-8-1) and 30 epochs - 6 months

# Let's create a keras sequential model
model_128_64_64_8_8_1_30_6_months_mag_warped_0_2_1 = Sequential()

# Let's add two LSTM layer's to the model network
model_128_64_64_8_8_1_30_6_months_mag_warped_0_2_1.add(
    LSTM(
        128,
        return_sequences=True,
        input_shape=(x_train_6_months_mag_warped_0_2_1.shape[1], 1),
    )
)
model_128_64_64_8_8_1_30_6_months_mag_warped_0_2_1.add(LSTM(64, return_sequences=True))
model_128_64_64_8_8_1_30_6_months_mag_warped_0_2_1.add(LSTM(64, return_sequences=False))

# Let's add dense layer to the model network
model_128_64_64_8_8_1_30_6_months_mag_warped_0_2_1.add(Dense(8))
model_128_64_64_8_8_1_30_6_months_mag_warped_0_2_1.add(Dense(8))
model_128_64_64_8_8_1_30_6_months_mag_warped_0_2_1.add(Dense(1))

# Let's build the model using mean_squared_error as the loss function
# and mean_absolute_error as the evaluation metric during the compilation process
model_128_64_64_8_8_1_30_6_months_mag_warped_0_2_1.compile(
    optimizer="adam", loss="mean_squared_error", metrics=["mean_absolute_error"]
)

# Let's define the callbacks for Model saving and Early stopping
cb_check = keras.callbacks.ModelCheckpoint(
    filepath="model_128_64_64_8_8_1_30_6_months_mag_warped_0_2_1_checkpoint_filepath",
    save_best_only=True,
    monitor="val_loss",
)

cb_early = keras.callbacks.EarlyStopping(monitor="val_loss", patience=30)

# Let's fit the sequential model with input features and output label
history_128_64_64_8_8_1_30_6_months_mag_warped_0_2_1 = (
    model_128_64_64_8_8_1_30_6_months_mag_warped_0_2_1.fit(
        x_train_6_months_mag_warped_0_2_1,
        y_train_6_months,
        batch_size=16,
        epochs=30,
        callbacks=[cb_check, cb_early],
        validation_data=(x_valid_6_months, y_valid_6_months),
    )
)

Epoch 1/30
140/140 [==============================] - 38s 220ms/step - loss: 4.2035e-04 - mean_absolute_error: 0.0120 - val_loss: 4.1987e-04 - val_mean_absolute_error: 0.0174
Epoch 2/30
140/140 [==============================] - 24s 173ms/step - loss: 3.3166e-04 - mean_absolute_error: 0.0118 - val_loss: 4.3665e-04 - val_mean_absolute_error: 0.0175
Epoch 3/30
140/140 [==============================] - 29s 208ms/step - loss: 2.7026e-04 - mean_absolute_error: 0.0103 - val_loss: 3.5028e-04 - val_mean_absolute_error: 0.0158
Epoch 4/30
140/140 [==============================] - 24s 173ms/step - loss: 2.5898e-04 - mean_absolute_error: 0.0096 - val_loss: 5.2455e-04 - val_mean_absolute_error: 0.0185
Epoch 5/30
140/140 [==============================] - 29s 205ms/step - loss: 3.1840e-04 - mean_absolute_error: 0.0113 - val_loss: 3.3345e-04 - val_mean_absolute_error: 0.0145
Epoch 6/30
140/140 [==============================] - 29s 206ms/step - loss: 2.5217e-04 - mean_absolute_error: 0.0099 - val_loss: 2.8566e-04 - val_mean_absolute_error: 0.0140
Epoch 7/30
140/140 [==============================] - 57s 408ms/step - loss: 2.3509e-04 - mean_absolute_error: 0.0092 - val_loss: 2.6330e-04 - val_mean_absolute_error: 0.0136
Epoch 8/30
140/140 [==============================] - 24s 171ms/step - loss: 2.8618e-04 - mean_absolute_error: 0.0109 - val_loss: 3.5618e-04 - val_mean_absolute_error: 0.0155
Epoch 9/30
140/140 [==============================] - 28s 204ms/step - loss: 2.3567e-04 - mean_absolute_error: 0.0095 - val_loss: 2.4292e-04 - val_mean_absolute_error: 0.0129
Epoch 10/30
140/140 [==============================] - 24s 172ms/step - loss: 2.4855e-04 - mean_absolute_error: 0.0097 - val_loss: 6.2181e-04 - val_mean_absolute_error: 0.0201
Epoch 11/30
140/140 [==============================] - 25s 178ms/step - loss: 2.4909e-04 - mean_absolute_error: 0.0097 - val_loss: 2.6883e-04 - val_mean_absolute_error: 0.0136
Epoch 12/30
140/140 [==============================] - 31s 223ms/step - loss: 2.6650e-04 - mean_absolute_error: 0.0100 - val_loss: 2.3822e-04 - val_mean_absolute_error: 0.0124
Epoch 13/30
140/140 [==============================] - 25s 178ms/step - loss: 2.6033e-04 - mean_absolute_error: 0.0100 - val_loss: 4.4795e-04 - val_mean_absolute_error: 0.0168
Epoch 14/30
140/140 [==============================] - 24s 174ms/step - loss: 2.6049e-04 - mean_absolute_error: 0.0100 - val_loss: 2.5140e-04 - val_mean_absolute_error: 0.0133
Epoch 15/30
140/140 [==============================] - 32s 226ms/step - loss: 2.5266e-04 - mean_absolute_error: 0.0098 - val_loss: 2.1485e-04 - val_mean_absolute_error: 0.0120
Epoch 16/30
140/140 [==============================] - 25s 176ms/step - loss: 2.4199e-04 - mean_absolute_error: 0.0097 - val_loss: 2.1946e-04 - val_mean_absolute_error: 0.0123
Epoch 17/30
140/140 [==============================] - 25s 177ms/step - loss: 2.5728e-04 - mean_absolute_error: 0.0099 - val_loss: 2.8578e-04 - val_mean_absolute_error: 0.0137
Epoch 18/30
140/140 [==============================] - 24s 174ms/step - loss: 2.2197e-04 - mean_absolute_error: 0.0086 - val_loss: 8.4430e-04 - val_mean_absolute_error: 0.0257
Epoch 19/30
140/140 [==============================] - 24s 172ms/step - loss: 2.2330e-04 - mean_absolute_error: 0.0091 - val_loss: 7.5578e-04 - val_mean_absolute_error: 0.0229
Epoch 20/30
140/140 [==============================] - 26s 185ms/step - loss: 2.4457e-04 - mean_absolute_error: 0.0094 - val_loss: 5.0630e-04 - val_mean_absolute_error: 0.0187
Epoch 21/30
140/140 [==============================] - 24s 173ms/step - loss: 2.4857e-04 - mean_absolute_error: 0.0098 - val_loss: 2.2819e-04 - val_mean_absolute_error: 0.0124
Epoch 22/30
140/140 [==============================] - 24s 173ms/step - loss: 2.2330e-04 - mean_absolute_error: 0.0088 - val_loss: 3.5108e-04 - val_mean_absolute_error: 0.0149
Epoch 23/30
140/140 [==============================] - 24s 174ms/step - loss: 2.4924e-04 - mean_absolute_error: 0.0097 - val_loss: 4.2811e-04 - val_mean_absolute_error: 0.0163
Epoch 24/30
140/140 [==============================] - 25s 178ms/step - loss: 2.4017e-04 - mean_absolute_error: 0.0091 - val_loss: 5.0828e-04 - val_mean_absolute_error: 0.0183
Epoch 25/30
140/140 [==============================] - 31s 219ms/step - loss: 2.3485e-04 - mean_absolute_error: 0.0094 - val_loss: 1.8743e-04 - val_mean_absolute_error: 0.0113
Epoch 26/30
140/140 [==============================] - 25s 177ms/step - loss: 2.3520e-04 - mean_absolute_error: 0.0089 - val_loss: 2.0758e-04 - val_mean_absolute_error: 0.0118
Epoch 27/30
140/140 [==============================] - 25s 179ms/step - loss: 2.3992e-04 - mean_absolute_error: 0.0095 - val_loss: 3.7875e-04 - val_mean_absolute_error: 0.0157
Epoch 28/30
140/140 [==============================] - 29s 206ms/step - loss: 2.2680e-04 - mean_absolute_error: 0.0089 - val_loss: 4.4801e-04 - val_mean_absolute_error: 0.0169
Epoch 29/30
140/140 [==============================] - 31s 221ms/step - loss: 2.2866e-04 - mean_absolute_error: 0.0089 - val_loss: 1.8150e-04 - val_mean_absolute_error: 0.0112
Epoch 30/30
140/140 [==============================] - 25s 177ms/step - loss: 2.4554e-04 - mean_absolute_error: 0.0092 - val_loss: 4.3419e-04 - val_mean_absolute_error: 0.0167

# Let's plot the mean_squared_error and mean_absolute_error using model history on the entire dataset
plot_loss_and_mae_validation(
    model_name="model_128_64_64_8_8_1_30_6_months_mag_warped_0_2_1",
    model_history=history_128_64_64_8_8_1_30_6_months_mag_warped_0_2_1,
)

Observations

Using six months of data as a window to predict the stock price and applying warping by constant amounts (0.2 and 4) to the input training samples, the training loss and mean absolute error are gradually decreasing. In contrast, the validation loss and mean absolute error are fluctuating, i.e., both increasing and decreasing. The observed validation mean absolute error is greater than that obtained using a high-performance model with three LSTM layers and three dense layers. It is also greater than when using noise levels of 0.005, 0.03, and 0.01 and scale constants of 0.01, 0.1, and 0.3 with the same six-month data window. One option to potentially decrease the validation mean absolute error further is to adjust the scaling magnitude.

• 3 LSTM(128-64-64) 3 Dense(8-8-1) - 6 months - 0.2 Warping
• Epochs: 30
• Batch Size: 16
• Training Loss: 2.4554e-04
• Validation Loss: 4.3419e-04
• Training MAE: 0.0092
• Validation MAE: 0.0167

sigma = 0.2, knot = 4

# Let's warp each time series of six-month training data window by constant amount
x_train_6_months_mag_warped_0_2_4 = magnitude_warp(x_train_6_months, sigma=0.2, knot=4)

Model Building with 3 LSTM and 3 Dense layers (128-64-64-8-8-1) and 30 epochs - 6 months

# Let's create a keras sequential model
model_128_64_64_8_8_1_30_6_months_mag_warped_0_2_4 = Sequential()

# Let's add two LSTM layer's to the model network
model_128_64_64_8_8_1_30_6_months_mag_warped_0_2_4.add(
    LSTM(
        128,
        return_sequences=True,
        input_shape=(x_train_6_months_mag_warped_0_2_4.shape[1], 1),
    )
)
model_128_64_64_8_8_1_30_6_months_mag_warped_0_2_4.add(LSTM(64, return_sequences=True))
model_128_64_64_8_8_1_30_6_months_mag_warped_0_2_4.add(LSTM(64, return_sequences=False))

# Let's add dense layer to the model network
model_128_64_64_8_8_1_30_6_months_mag_warped_0_2_4.add(Dense(8))
model_128_64_64_8_8_1_30_6_months_mag_warped_0_2_4.add(Dense(8))
model_128_64_64_8_8_1_30_6_months_mag_warped_0_2_4.add(Dense(1))

# Let's build the model using mean_squared_error as the loss function
# and mean_absolute_error as the evaluation metric during the compilation process
model_128_64_64_8_8_1_30_6_months_mag_warped_0_2_4.compile(
    optimizer="adam", loss="mean_squared_error", metrics=["mean_absolute_error"]
)

# Let's define the callbacks for Model saving and Early stopping
cb_check = keras.callbacks.ModelCheckpoint(
    filepath="model_128_64_64_8_8_1_30_6_months_mag_warped_0_2_4_checkpoint_filepath",
    save_best_only=True,
    monitor="val_loss",
)

cb_early = keras.callbacks.EarlyStopping(monitor="val_loss", patience=30)

# Let's fit the sequential model with input features and output label
history_128_64_64_8_8_1_30_6_months_mag_warped_0_2_4 = (
    model_128_64_64_8_8_1_30_6_months_mag_warped_0_2_4.fit(
        x_train_6_months_mag_warped_0_2_4,
        y_train_6_months,
        batch_size=16,
        epochs=30,
        callbacks=[cb_check, cb_early],
        validation_data=(x_valid_6_months, y_valid_6_months),
    )
)

Epoch 1/30
140/140 [==============================] - 37s 227ms/step - loss: 5.1133e-04 - mean_absolute_error: 0.0132 - val_loss: 5.0108e-04 - val_mean_absolute_error: 0.0176
Epoch 2/30
140/140 [==============================] - 48s 341ms/step - loss: 2.8219e-04 - mean_absolute_error: 0.0113 - val_loss: 3.5042e-04 - val_mean_absolute_error: 0.0151
Epoch 3/30
140/140 [==============================] - 27s 189ms/step - loss: 2.6562e-04 - mean_absolute_error: 0.0101 - val_loss: 5.7363e-04 - val_mean_absolute_error: 0.0190
Epoch 4/30
140/140 [==============================] - 30s 212ms/step - loss: 2.3107e-04 - mean_absolute_error: 0.0095 - val_loss: 2.9615e-04 - val_mean_absolute_error: 0.0137
Epoch 5/30
140/140 [==============================] - 25s 176ms/step - loss: 2.1952e-04 - mean_absolute_error: 0.0092 - val_loss: 4.6149e-04 - val_mean_absolute_error: 0.0172
Epoch 6/30
140/140 [==============================] - 24s 173ms/step - loss: 2.5827e-04 - mean_absolute_error: 0.0105 - val_loss: 4.8862e-04 - val_mean_absolute_error: 0.0182
Epoch 7/30
140/140 [==============================] - 31s 222ms/step - loss: 2.2015e-04 - mean_absolute_error: 0.0095 - val_loss: 2.2758e-04 - val_mean_absolute_error: 0.0126
Epoch 8/30
140/140 [==============================] - 20s 139ms/step - loss: 1.9910e-04 - mean_absolute_error: 0.0086 - val_loss: 2.4639e-04 - val_mean_absolute_error: 0.0131
Epoch 9/30
140/140 [==============================] - 20s 140ms/step - loss: 2.2535e-04 - mean_absolute_error: 0.0093 - val_loss: 1.9858e-04 - val_mean_absolute_error: 0.0118
Epoch 10/30
140/140 [==============================] - 20s 141ms/step - loss: 2.0921e-04 - mean_absolute_error: 0.0093 - val_loss: 1.9125e-04 - val_mean_absolute_error: 0.0115
Epoch 11/30
140/140 [==============================] - 17s 117ms/step - loss: 1.9461e-04 - mean_absolute_error: 0.0082 - val_loss: 2.5920e-04 - val_mean_absolute_error: 0.0136
Epoch 12/30
140/140 [==============================] - 20s 140ms/step - loss: 1.9307e-04 - mean_absolute_error: 0.0083 - val_loss: 1.7574e-04 - val_mean_absolute_error: 0.0110
Epoch 13/30
140/140 [==============================] - 17s 117ms/step - loss: 2.0538e-04 - mean_absolute_error: 0.0091 - val_loss: 2.0397e-04 - val_mean_absolute_error: 0.0120
Epoch 14/30
140/140 [==============================] - 16s 117ms/step - loss: 1.9224e-04 - mean_absolute_error: 0.0084 - val_loss: 3.5324e-04 - val_mean_absolute_error: 0.0152
Epoch 15/30
140/140 [==============================] - 16s 117ms/step - loss: 2.3077e-04 - mean_absolute_error: 0.0095 - val_loss: 2.0924e-04 - val_mean_absolute_error: 0.0116
Epoch 16/30
140/140 [==============================] - 16s 116ms/step - loss: 2.2111e-04 - mean_absolute_error: 0.0091 - val_loss: 2.1788e-04 - val_mean_absolute_error: 0.0120
Epoch 17/30
140/140 [==============================] - 17s 118ms/step - loss: 2.1874e-04 - mean_absolute_error: 0.0092 - val_loss: 1.9817e-04 - val_mean_absolute_error: 0.0116
Epoch 18/30
140/140 [==============================] - 17s 123ms/step - loss: 2.0718e-04 - mean_absolute_error: 0.0088 - val_loss: 2.1611e-04 - val_mean_absolute_error: 0.0122
Epoch 19/30
140/140 [==============================] - 17s 119ms/step - loss: 2.0630e-04 - mean_absolute_error: 0.0090 - val_loss: 5.4006e-04 - val_mean_absolute_error: 0.0195
Epoch 20/30
140/140 [==============================] - 16s 117ms/step - loss: 2.1010e-04 - mean_absolute_error: 0.0090 - val_loss: 4.9057e-04 - val_mean_absolute_error: 0.0181
Epoch 21/30
140/140 [==============================] - 16s 116ms/step - loss: 1.9400e-04 - mean_absolute_error: 0.0087 - val_loss: 2.1372e-04 - val_mean_absolute_error: 0.0122
Epoch 22/30
140/140 [==============================] - 16s 117ms/step - loss: 1.7560e-04 - mean_absolute_error: 0.0079 - val_loss: 3.6390e-04 - val_mean_absolute_error: 0.0154
Epoch 23/30
140/140 [==============================] - 16s 116ms/step - loss: 1.6784e-04 - mean_absolute_error: 0.0078 - val_loss: 2.3527e-04 - val_mean_absolute_error: 0.0131
Epoch 24/30
140/140 [==============================] - 16s 116ms/step - loss: 1.9659e-04 - mean_absolute_error: 0.0089 - val_loss: 2.7046e-04 - val_mean_absolute_error: 0.0134
Epoch 25/30
140/140 [==============================] - 16s 116ms/step - loss: 2.0677e-04 - mean_absolute_error: 0.0089 - val_loss: 3.3255e-04 - val_mean_absolute_error: 0.0148
Epoch 26/30
140/140 [==============================] - 20s 145ms/step - loss: 1.9183e-04 - mean_absolute_error: 0.0084 - val_loss: 1.7091e-04 - val_mean_absolute_error: 0.0110
Epoch 27/30
140/140 [==============================] - 16s 116ms/step - loss: 1.8543e-04 - mean_absolute_error: 0.0083 - val_loss: 2.7350e-04 - val_mean_absolute_error: 0.0141
Epoch 28/30
140/140 [==============================] - 16s 116ms/step - loss: 1.6465e-04 - mean_absolute_error: 0.0079 - val_loss: 3.9354e-04 - val_mean_absolute_error: 0.0162
Epoch 29/30
140/140 [==============================] - 16s 117ms/step - loss: 1.6685e-04 - mean_absolute_error: 0.0080 - val_loss: 0.0014 - val_mean_absolute_error: 0.0334
Epoch 30/30
140/140 [==============================] - 16s 117ms/step - loss: 1.8183e-04 - mean_absolute_error: 0.0080 - val_loss: 4.6530e-04 - val_mean_absolute_error: 0.0176

# Let's plot the mean_squared_error and mean_absolute_error using model history on the entire dataset
plot_loss_and_mae_validation(
    model_name="model_128_64_64_8_8_1_30_6_months_mag_warped_0_2_4",
    model_history=history_128_64_64_8_8_1_30_6_months_mag_warped_0_2_4,
)

Observations

Using six months of data as a window to predict the stock price and applying warping by constant amounts (0.2 and 4) to the input training samples, the training loss and mean absolute error are gradually decreasing. In contrast, the validation loss and mean absolute error are fluctuating, i.e., both increasing and decreasing. The observed validation mean absolute error is greater than that obtained using a high-performance model with three LSTM layers and three dense layers. It is also greater than when using noise levels of 0.005, 0.03, and 0.01 and scale constants of 0.01, 0.1, and 0.3 with the same six-month data window. One option to potentially decrease the validation mean absolute error further is to adjust the scaling magnitude.

• 3 LSTM(128-64-64) 3 Dense(8-8-1) - 6 months - 0.2 Warping
• Epochs: 30
• Batch Size: 16
• Training Loss: 2.4554e-04
• Validation Loss: 4.3419e-04
• Training MAE: 0.0092
• Validation MAE: 0.0167
  
  

# Let's define the list of models that are used for validation and comparison.
model_validation_list_augmentation = {
    "model_128_64_64_8_8_1_30_6_months_validation_0_0_0_5_noise": model_128_64_64_8_8_1_30_6_months_validation_0_0_0_5_noise,
    "model_128_64_64_8_8_1_30_6_months_validation_0_0_1_noise": model_128_64_64_8_8_1_30_6_months_validation_0_0_1_noise,
    "model_128_64_64_8_8_1_30_6_months_validation_0_0_3_noise": model_128_64_64_8_8_1_30_6_months_validation_0_0_3_noise,
    "model_128_64_64_8_8_1_30_6_months_validation_0_0_1_scaled": model_128_64_64_8_8_1_30_6_months_validation_0_0_1_scaled,
    "model_128_64_64_8_8_1_30_6_months_validation_0_1_scaled": model_128_64_64_8_8_1_30_6_months_validation_0_1_scaled,
    "model_128_64_64_8_8_1_30_6_months_validation_0_3_scaled": model_128_64_64_8_8_1_30_6_months_validation_0_3_scaled,
    "model_128_64_64_8_8_1_30_6_months_mag_warped_0_2_1": model_128_64_64_8_8_1_30_6_months_mag_warped_0_2_1,
    "model_128_64_64_8_8_1_30_6_months_mag_warped_0_2_4": model_128_64_64_8_8_1_30_6_months_mag_warped_0_2_1,
}

model_validation_list_augmentation_plot = {
    "model_128_64_64_8_8_1_30_6_months_validation_0_0_1_scaled": model_128_64_64_8_8_1_30_6_months_validation_0_0_1_scaled,
}

validation_testing_6_months_augmentation = []

for model_name in model_validation_list_augmentation_plot.keys():
    if "6_months" in model_name:
        model = keras.models.load_model(f"{model_name}_checkpoint_filepath")
        six_months_metrics_dict_augmentation = get_metrics(period=6, model=model)
        plot_metrics_and_predictions(
            model_name=model_name,
            actual_prices=six_months_metrics_dict_augmentation.get(
                "valid_actual_prices"
            ),
            predicted_prices=six_months_metrics_dict_augmentation.get(
                "valid_predicted_prices"
            ),
        )
        validation_testing_6_months_augmentation.append(
            {
                "Model Configuration": model_name,
                "period": "6 months",
                "Validation MAE": six_months_metrics_dict_augmentation.get(
                    "validation_mae"
                ),
                "Validation MAPE": six_months_metrics_dict_augmentation.get(
                    "validation_mape"
                ),
                "Test MAE": six_months_metrics_dict_augmentation.get("test_mae"),
                "Test MAPE": six_months_metrics_dict_augmentation.get("test_mape"),
            }
        )

Printing Validation MSE and MAE for all the augmented models - 6 months

# Let's convert the validation_testing list of dictionaries into a pandas data_frame
validation_testing_6_months_augmented_df = pd.DataFrame(
    validation_testing_6_months_augmentation
)
validation_testing_6_months_augmented_df

	Model Configuration	period	Validation MAE	Validation MAPE	Test MAE	Test MAPE
0	model_128_64_64_8_8_1_30_6_months_validation_0...	6 months	6.802560	0.044311	19.602779	0.029615
1	model_128_64_64_8_8_1_30_6_months_validation_0...	6 months	7.872104	0.050886	38.037239	0.052338
2	model_128_64_64_8_8_1_30_6_months_validation_0...	6 months	12.814412	0.082905	92.741540	0.127254
3	model_128_64_64_8_8_1_30_6_months_validation_0...	6 months	6.142313	0.039434	18.831326	0.027169
4	model_128_64_64_8_8_1_30_6_months_validation_0...	6 months	6.249270	0.040435	99.480830	0.148395
5	model_128_64_64_8_8_1_30_6_months_validation_0...	6 months	7.824560	0.049986	258.880185	0.359946
6	model_128_64_64_8_8_1_30_6_months_mag_warped_0...	6 months	10.648151	0.070893	203.769306	0.281648
7	model_128_64_64_8_8_1_30_6_months_mag_warped_0...	6 months	10.371709	0.069998	204.302959	0.280349

Observations

---

📊The above table, which shows comparisons of different LSTM model configurations with added noise and scaling magnitude augmentations, represents a distinct noise/scaling constant in each row. These models were trained using a 6-month window with 3 LSTM and 3 Dense layers to predict daily stock prices. From the table, it is evident that the LSTM model trained with a scaling constant of 0.001 and a 6-month window outperformed those trained with other noise and scaling constant values 📈

💡The best model among the different configurations of LSTM layers and neurons is the one with three LSTM layers (128, 64, and 64 neurons, respectively) and three dense layers (8, 8, and 1 neurons, respectively) with 0.0.1 scaling constant🏆

Effects of Regularization

from keras.layers import BatchNormalization, Dropout

Model Building with 3 LSTM and 3 Dense layers (128-64-64-8-8-1) and 30 epochs - 6 months - Batch Normalization (1 Layer)

# Let's create a keras sequential model
model_128_64_64_8_8_1_30_6_months_1_layer_batch_normalization = Sequential()

# Let's add two LSTM layer's to the model network
model_128_64_64_8_8_1_30_6_months_1_layer_batch_normalization.add(
    LSTM(128, return_sequences=True, input_shape=(x_train_6_months.shape[1], 1))
)
# Let's add Batch Normalization Layer
model_128_64_64_8_8_1_30_6_months_1_layer_batch_normalization.add(BatchNormalization())

model_128_64_64_8_8_1_30_6_months_1_layer_batch_normalization.add(
    LSTM(64, return_sequences=True)
)
model_128_64_64_8_8_1_30_6_months_1_layer_batch_normalization.add(
    LSTM(64, return_sequences=False)
)

# Let's add dense layer to the model network
model_128_64_64_8_8_1_30_6_months_1_layer_batch_normalization.add(Dense(8))
model_128_64_64_8_8_1_30_6_months_1_layer_batch_normalization.add(Dense(8))
model_128_64_64_8_8_1_30_6_months_1_layer_batch_normalization.add(Dense(1))

# Let's build the model using mean_squared_error as the loss function
# and mean_absolute_error as the evaluation metric during the compilation process
model_128_64_64_8_8_1_30_6_months_1_layer_batch_normalization.compile(
    optimizer="adam", loss="mean_squared_error", metrics=["mean_absolute_error"]
)

# Let's define the callbacks for Model saving and Early stopping
cb_check = keras.callbacks.ModelCheckpoint(
    filepath="model_128_64_64_8_8_1_30_6_months_1_layer_batch_normalization_checkpoint_filepath",
    save_best_only=True,
    monitor="val_loss",
)

cb_early = keras.callbacks.EarlyStopping(monitor="val_loss", patience=30)

# Let's fit the sequential model with input features and output label
history_128_64_64_8_8_1_30_6_months_1_layer_batch_normalization = (
    model_128_64_64_8_8_1_30_6_months_1_layer_batch_normalization.fit(
        x_train_6_months,
        y_train_6_months,
        batch_size=16,
        epochs=30,
        callbacks=[cb_check, cb_early],
        validation_data=(x_valid_6_months, y_valid_6_months),
    )
)

Epoch 1/30
140/140 [==============================] - 40s 243ms/step - loss: 0.0010 - mean_absolute_error: 0.0166 - val_loss: 0.0121 - val_mean_absolute_error: 0.1059
Epoch 2/30
140/140 [==============================] - 25s 177ms/step - loss: 2.3863e-04 - mean_absolute_error: 0.0123 - val_loss: 0.0122 - val_mean_absolute_error: 0.1069
Epoch 3/30
140/140 [==============================] - 31s 219ms/step - loss: 2.8358e-04 - mean_absolute_error: 0.0121 - val_loss: 0.0105 - val_mean_absolute_error: 0.0990
Epoch 4/30
140/140 [==============================] - 29s 210ms/step - loss: 2.4691e-04 - mean_absolute_error: 0.0122 - val_loss: 0.0094 - val_mean_absolute_error: 0.0949
Epoch 5/30
140/140 [==============================] - 30s 212ms/step - loss: 2.1216e-04 - mean_absolute_error: 0.0112 - val_loss: 0.0010 - val_mean_absolute_error: 0.0285
Epoch 6/30
140/140 [==============================] - 30s 213ms/step - loss: 2.0581e-04 - mean_absolute_error: 0.0111 - val_loss: 1.7989e-04 - val_mean_absolute_error: 0.0111
Epoch 7/30
140/140 [==============================] - 31s 217ms/step - loss: 2.0774e-04 - mean_absolute_error: 0.0112 - val_loss: 1.5129e-04 - val_mean_absolute_error: 0.0100
Epoch 8/30
140/140 [==============================] - 25s 179ms/step - loss: 1.7819e-04 - mean_absolute_error: 0.0105 - val_loss: 8.7958e-04 - val_mean_absolute_error: 0.0273
Epoch 9/30
140/140 [==============================] - 25s 177ms/step - loss: 1.7928e-04 - mean_absolute_error: 0.0105 - val_loss: 3.6408e-04 - val_mean_absolute_error: 0.0167
Epoch 10/30
140/140 [==============================] - 25s 175ms/step - loss: 1.6902e-04 - mean_absolute_error: 0.0097 - val_loss: 4.7440e-04 - val_mean_absolute_error: 0.0179
Epoch 11/30
140/140 [==============================] - 25s 181ms/step - loss: 1.5656e-04 - mean_absolute_error: 0.0099 - val_loss: 3.0469e-04 - val_mean_absolute_error: 0.0149
Epoch 12/30
140/140 [==============================] - 25s 176ms/step - loss: 1.7338e-04 - mean_absolute_error: 0.0099 - val_loss: 6.1971e-04 - val_mean_absolute_error: 0.0222
Epoch 13/30
140/140 [==============================] - 25s 180ms/step - loss: 1.5553e-04 - mean_absolute_error: 0.0099 - val_loss: 3.0889e-04 - val_mean_absolute_error: 0.0149
Epoch 14/30
140/140 [==============================] - 30s 211ms/step - loss: 2.0734e-04 - mean_absolute_error: 0.0112 - val_loss: 1.1748e-04 - val_mean_absolute_error: 0.0091
Epoch 15/30
140/140 [==============================] - 26s 184ms/step - loss: 2.2893e-04 - mean_absolute_error: 0.0116 - val_loss: 2.7579e-04 - val_mean_absolute_error: 0.0137
Epoch 16/30
140/140 [==============================] - 26s 183ms/step - loss: 1.6689e-04 - mean_absolute_error: 0.0103 - val_loss: 1.7929e-04 - val_mean_absolute_error: 0.0106
Epoch 17/30
140/140 [==============================] - 25s 179ms/step - loss: 1.4581e-04 - mean_absolute_error: 0.0091 - val_loss: 8.2892e-04 - val_mean_absolute_error: 0.0273
Epoch 18/30
140/140 [==============================] - 49s 354ms/step - loss: 1.6912e-04 - mean_absolute_error: 0.0095 - val_loss: 1.0427e-04 - val_mean_absolute_error: 0.0088
Epoch 19/30
140/140 [==============================] - 25s 174ms/step - loss: 1.6871e-04 - mean_absolute_error: 0.0095 - val_loss: 1.7604e-04 - val_mean_absolute_error: 0.0107
Epoch 20/30
140/140 [==============================] - 24s 173ms/step - loss: 1.5536e-04 - mean_absolute_error: 0.0096 - val_loss: 2.8276e-04 - val_mean_absolute_error: 0.0143
Epoch 21/30
140/140 [==============================] - 31s 224ms/step - loss: 1.3752e-04 - mean_absolute_error: 0.0090 - val_loss: 9.0824e-05 - val_mean_absolute_error: 0.0078
Epoch 22/30
140/140 [==============================] - 25s 177ms/step - loss: 2.2743e-04 - mean_absolute_error: 0.0108 - val_loss: 0.0013 - val_mean_absolute_error: 0.0345
Epoch 23/30
140/140 [==============================] - 24s 174ms/step - loss: 1.1503e-04 - mean_absolute_error: 0.0084 - val_loss: 1.7533e-04 - val_mean_absolute_error: 0.0107
Epoch 24/30
140/140 [==============================] - 24s 174ms/step - loss: 1.3926e-04 - mean_absolute_error: 0.0088 - val_loss: 2.1301e-04 - val_mean_absolute_error: 0.0121
Epoch 25/30
140/140 [==============================] - 25s 177ms/step - loss: 1.3673e-04 - mean_absolute_error: 0.0088 - val_loss: 0.0015 - val_mean_absolute_error: 0.0369
Epoch 26/30
140/140 [==============================] - 24s 174ms/step - loss: 1.6524e-04 - mean_absolute_error: 0.0092 - val_loss: 2.8491e-04 - val_mean_absolute_error: 0.0147
Epoch 27/30
140/140 [==============================] - 30s 217ms/step - loss: 1.4455e-04 - mean_absolute_error: 0.0090 - val_loss: 7.2980e-05 - val_mean_absolute_error: 0.0072
Epoch 28/30
140/140 [==============================] - 25s 176ms/step - loss: 1.5281e-04 - mean_absolute_error: 0.0089 - val_loss: 7.9599e-04 - val_mean_absolute_error: 0.0267
Epoch 29/30
140/140 [==============================] - 24s 174ms/step - loss: 1.4557e-04 - mean_absolute_error: 0.0089 - val_loss: 7.6529e-05 - val_mean_absolute_error: 0.0074
Epoch 30/30
140/140 [==============================] - 25s 175ms/step - loss: 1.0175e-04 - mean_absolute_error: 0.0075 - val_loss: 2.7959e-04 - val_mean_absolute_error: 0.0142

# Let's plot the mean_squared_error and mean_absolute_error using model history on the entire dataset
plot_loss_and_mae_validation(
    model_name="model_128_64_64_8_8_1_30_6_months_1_layer_batch_normalization",
    model_history=history_128_64_64_8_8_1_30_6_months_1_layer_batch_normalization,
)

Observations

Using six months of data as a window to predict stock prices and adding one batch normalization layer to the input training samples, the training loss and mean absolute error are decreasing gradually. In contrast, the validation loss and mean absolute error are fluctuating, i.e., both increasing and decreasing. The observed validation mean absolute error is greater than that obtained using a high-performance model with 3 LSTM layers and 3 Dense layers. It is significantly less than when using noise levels of 0.005, 0.03, and 0.01 with the same six-month data window and less than when using scaling constants of 0.1, and 0.3. One option to potentially decrease the validation mean absolute error further is to adjust the scaling magnitude.

• 3 LSTM(128-64-64) 3 Dense(8-8-1) - 6 months - 1 Batch Normalization layer
• Epochs: 30
• Batch Size: 16
• Training Loss: 1.4557e-04
• Validation Loss: 2.7959e-04
• Training MAE: 0.0075
• Validation MAE: 0.0074

Model Building with 3 LSTM and 3 Dense layers (128-64-64-8-8-1) and 30 epochs - 6 months - Batch Normalization (2 Layers)

# Let's create a keras sequential model
model_128_64_64_8_8_1_30_6_months_2_layer_batch_normalization = Sequential()

# Let's add two LSTM layer's to the model network
model_128_64_64_8_8_1_30_6_months_2_layer_batch_normalization.add(
    LSTM(128, return_sequences=True, input_shape=(x_train_6_months.shape[1], 1))
)
# Let's add Batch Normalization Layer
model_128_64_64_8_8_1_30_6_months_2_layer_batch_normalization.add(BatchNormalization())
model_128_64_64_8_8_1_30_6_months_2_layer_batch_normalization.add(
    LSTM(64, return_sequences=True)
)
model_128_64_64_8_8_1_30_6_months_2_layer_batch_normalization.add(
    LSTM(64, return_sequences=False)
)
# Let's add Batch Normalization Layer
model_128_64_64_8_8_1_30_6_months_2_layer_batch_normalization.add(BatchNormalization())

# Let's add dense layer to the model network
model_128_64_64_8_8_1_30_6_months_2_layer_batch_normalization.add(Dense(8))
model_128_64_64_8_8_1_30_6_months_2_layer_batch_normalization.add(Dense(8))
model_128_64_64_8_8_1_30_6_months_2_layer_batch_normalization.add(Dense(1))

# Let's build the model using mean_squared_error as the loss function
# and mean_absolute_error as the evaluation metric during the compilation process
model_128_64_64_8_8_1_30_6_months_2_layer_batch_normalization.compile(
    optimizer="adam", loss="mean_squared_error", metrics=["mean_absolute_error"]
)

# Let's define the callbacks for Model saving and Early stopping
cb_check = keras.callbacks.ModelCheckpoint(
    filepath="model_128_64_64_8_8_1_30_6_months_2_layer_batch_normalization_checkpoint_filepath",
    save_best_only=True,
    monitor="val_loss",
)

cb_early = keras.callbacks.EarlyStopping(monitor="val_loss", patience=30)

# Let's fit the sequential model with input features and output label
history_128_64_64_8_8_1_30_6_months_2_layer_batch_normalization = (
    model_128_64_64_8_8_1_30_6_months_2_layer_batch_normalization.fit(
        x_train_6_months,
        y_train_6_months,
        batch_size=16,
        epochs=30,
        callbacks=[cb_check, cb_early],
        validation_data=(x_valid_6_months, y_valid_6_months),
    )
)

Epoch 1/30
140/140 [==============================] - 36s 220ms/step - loss: 0.0503 - mean_absolute_error: 0.0684 - val_loss: 0.0121 - val_mean_absolute_error: 0.1060
Epoch 2/30
140/140 [==============================] - 30s 215ms/step - loss: 7.6245e-04 - mean_absolute_error: 0.0218 - val_loss: 0.0082 - val_mean_absolute_error: 0.0866
Epoch 3/30
140/140 [==============================] - 32s 231ms/step - loss: 6.2507e-04 - mean_absolute_error: 0.0198 - val_loss: 0.0058 - val_mean_absolute_error: 0.0724
Epoch 4/30
140/140 [==============================] - 26s 188ms/step - loss: 4.6379e-04 - mean_absolute_error: 0.0165 - val_loss: 0.0241 - val_mean_absolute_error: 0.1526
Epoch 5/30
140/140 [==============================] - 32s 226ms/step - loss: 6.4341e-04 - mean_absolute_error: 0.0199 - val_loss: 3.7768e-04 - val_mean_absolute_error: 0.0158
Epoch 6/30
140/140 [==============================] - 25s 180ms/step - loss: 4.8374e-04 - mean_absolute_error: 0.0175 - val_loss: 0.0013 - val_mean_absolute_error: 0.0333
Epoch 7/30
140/140 [==============================] - 26s 184ms/step - loss: 5.7532e-04 - mean_absolute_error: 0.0187 - val_loss: 0.0101 - val_mean_absolute_error: 0.0999
Epoch 8/30
140/140 [==============================] - 27s 192ms/step - loss: 5.1601e-04 - mean_absolute_error: 0.0173 - val_loss: 5.1320e-04 - val_mean_absolute_error: 0.0197
Epoch 9/30
140/140 [==============================] - 26s 187ms/step - loss: 6.7805e-04 - mean_absolute_error: 0.0203 - val_loss: 0.0034 - val_mean_absolute_error: 0.0575
Epoch 10/30
140/140 [==============================] - 25s 178ms/step - loss: 8.0772e-04 - mean_absolute_error: 0.0213 - val_loss: 0.0037 - val_mean_absolute_error: 0.0574
Epoch 11/30
140/140 [==============================] - 26s 186ms/step - loss: 4.6899e-04 - mean_absolute_error: 0.0174 - val_loss: 0.0024 - val_mean_absolute_error: 0.0476
Epoch 12/30
140/140 [==============================] - 25s 180ms/step - loss: 5.3390e-04 - mean_absolute_error: 0.0180 - val_loss: 0.0058 - val_mean_absolute_error: 0.0750
Epoch 13/30
140/140 [==============================] - 25s 180ms/step - loss: 6.7724e-04 - mean_absolute_error: 0.0198 - val_loss: 0.0106 - val_mean_absolute_error: 0.1019
Epoch 14/30
140/140 [==============================] - 30s 214ms/step - loss: 5.9419e-04 - mean_absolute_error: 0.0191 - val_loss: 2.5423e-04 - val_mean_absolute_error: 0.0134
Epoch 15/30
140/140 [==============================] - 27s 191ms/step - loss: 5.5482e-04 - mean_absolute_error: 0.0181 - val_loss: 8.6622e-04 - val_mean_absolute_error: 0.0276
Epoch 16/30
140/140 [==============================] - 26s 183ms/step - loss: 5.6977e-04 - mean_absolute_error: 0.0185 - val_loss: 0.0015 - val_mean_absolute_error: 0.0369
Epoch 17/30
140/140 [==============================] - 25s 182ms/step - loss: 9.6249e-04 - mean_absolute_error: 0.0242 - val_loss: 0.0065 - val_mean_absolute_error: 0.0782
Epoch 18/30
140/140 [==============================] - 26s 183ms/step - loss: 0.0012 - mean_absolute_error: 0.0252 - val_loss: 0.0011 - val_mean_absolute_error: 0.0317
Epoch 19/30
140/140 [==============================] - 26s 181ms/step - loss: 6.1031e-04 - mean_absolute_error: 0.0186 - val_loss: 8.4375e-04 - val_mean_absolute_error: 0.0277
Epoch 20/30
140/140 [==============================] - 25s 182ms/step - loss: 5.4432e-04 - mean_absolute_error: 0.0187 - val_loss: 5.3959e-04 - val_mean_absolute_error: 0.0215
Epoch 21/30
140/140 [==============================] - 47s 336ms/step - loss: 4.9373e-04 - mean_absolute_error: 0.0167 - val_loss: 2.3112e-04 - val_mean_absolute_error: 0.0130
Epoch 22/30
140/140 [==============================] - 25s 176ms/step - loss: 5.5027e-04 - mean_absolute_error: 0.0180 - val_loss: 0.0058 - val_mean_absolute_error: 0.0747
Epoch 23/30
140/140 [==============================] - 24s 173ms/step - loss: 5.7012e-04 - mean_absolute_error: 0.0186 - val_loss: 0.0019 - val_mean_absolute_error: 0.0428
Epoch 24/30
140/140 [==============================] - 24s 173ms/step - loss: 4.1368e-04 - mean_absolute_error: 0.0162 - val_loss: 0.0165 - val_mean_absolute_error: 0.1261
Epoch 25/30
140/140 [==============================] - 25s 176ms/step - loss: 7.9728e-04 - mean_absolute_error: 0.0211 - val_loss: 0.0017 - val_mean_absolute_error: 0.0401
Epoch 26/30
140/140 [==============================] - 24s 172ms/step - loss: 4.7229e-04 - mean_absolute_error: 0.0169 - val_loss: 0.0035 - val_mean_absolute_error: 0.0582
Epoch 27/30
140/140 [==============================] - 25s 177ms/step - loss: 7.3885e-04 - mean_absolute_error: 0.0204 - val_loss: 4.4959e-04 - val_mean_absolute_error: 0.0194
Epoch 28/30
140/140 [==============================] - 24s 173ms/step - loss: 8.4863e-04 - mean_absolute_error: 0.0219 - val_loss: 0.0047 - val_mean_absolute_error: 0.0676
Epoch 29/30
140/140 [==============================] - 24s 172ms/step - loss: 4.9952e-04 - mean_absolute_error: 0.0177 - val_loss: 0.0037 - val_mean_absolute_error: 0.0597
Epoch 30/30
140/140 [==============================] - 29s 209ms/step - loss: 4.6321e-04 - mean_absolute_error: 0.0165 - val_loss: 1.2961e-04 - val_mean_absolute_error: 0.0090

# Let's plot the mean_squared_error and mean_absolute_error using model history on the entire dataset
plot_loss_and_mae_validation(
    model_name="model_128_64_64_8_8_1_30_6_months_2_layer_batch_normalization",
    model_history=history_128_64_64_8_8_1_30_6_months_2_layer_batch_normalization,
)

Observations

Using six months of data as a window to predict stock prices and adding one batch normalization layer to the input training samples, the training loss and mean absolute error are decreasing gradually. In contrast, the validation loss and mean absolute error are fluctuating, i.e., both increasing and decreasing. The observed validation mean absolute error is greater than that obtained using a high-performance model with 3 LSTM layers and 3 Dense layers. It is significantly less than when using noise levels of 0.005, 0.03, and 0.01 with the same six-month data window and less than when using scaling constants of 0.01, 0.1, and 0.3. One option to potentially decrease the validation mean absolute error further is to adjust the scaling magnitude.

• 3 LSTM(128-64-64) 3 Dense(8-8-1) - 6 months - 2 Batch Normalization layer
• Epochs: 30
• Batch Size: 16
• Training Loss: 4.6321e-04
• Validation Loss: 1.2961e-04
• Training MAE: 0.0165
• Validation MAE: 0.0090

Model Building with 3 LSTM and 3 Dense layers (128-64-64-8-8-1) and 30 epochs - 6 months - Dropout (1 Layer)

# Let's create a keras sequential model
model_128_64_64_8_8_1_30_6_months_1_layer_dropout = Sequential()

# Let's add two LSTM layer's to the model network
model_128_64_64_8_8_1_30_6_months_1_layer_dropout.add(
    LSTM(128, return_sequences=True, input_shape=(x_train_6_months.shape[1], 1))
)
# Let's add Dropout Layer
model_128_64_64_8_8_1_30_6_months_1_layer_dropout.add(Dropout(0.2))
model_128_64_64_8_8_1_30_6_months_1_layer_dropout.add(LSTM(64, return_sequences=True))
model_128_64_64_8_8_1_30_6_months_1_layer_dropout.add(LSTM(64, return_sequences=False))

# Let's add dense layer to the model network
model_128_64_64_8_8_1_30_6_months_1_layer_dropout.add(Dense(8))
model_128_64_64_8_8_1_30_6_months_1_layer_dropout.add(Dense(8))
model_128_64_64_8_8_1_30_6_months_1_layer_dropout.add(Dense(1))

# Let's build the model using mean_squared_error as the loss function
# and mean_absolute_error as the evaluation metric during the compilation process
model_128_64_64_8_8_1_30_6_months_1_layer_dropout.compile(
    optimizer="adam", loss="mean_squared_error", metrics=["mean_absolute_error"]
)

# Let's define the callbacks for Model saving and Early stopping
cb_check = keras.callbacks.ModelCheckpoint(
    filepath="model_128_64_64_8_8_1_30_6_months_1_layer_dropout_checkpoint_filepath",
    save_best_only=True,
    monitor="val_loss",
)

cb_early = keras.callbacks.EarlyStopping(monitor="val_loss", patience=30)

# Let's fit the sequential model with input features and output label
history_128_64_64_8_8_1_30_6_months_1_layer_dropout = (
    model_128_64_64_8_8_1_30_6_months_1_layer_dropout.fit(
        x_train_6_months,
        y_train_6_months,
        batch_size=16,
        epochs=30,
        callbacks=[cb_check, cb_early],
        validation_data=(x_valid_6_months, y_valid_6_months),
    )
)

Epoch 1/30
140/140 [==============================] - 36s 216ms/step - loss: 1.7809e-04 - mean_absolute_error: 0.0077 - val_loss: 3.9082e-04 - val_mean_absolute_error: 0.0168
Epoch 2/30
140/140 [==============================] - 29s 208ms/step - loss: 6.1145e-05 - mean_absolute_error: 0.0050 - val_loss: 3.0907e-04 - val_mean_absolute_error: 0.0146
Epoch 3/30
140/140 [==============================] - 24s 174ms/step - loss: 5.6783e-05 - mean_absolute_error: 0.0051 - val_loss: 3.5496e-04 - val_mean_absolute_error: 0.0155
Epoch 4/30
140/140 [==============================] - 29s 206ms/step - loss: 3.8512e-05 - mean_absolute_error: 0.0041 - val_loss: 1.9902e-04 - val_mean_absolute_error: 0.0117
Epoch 5/30
140/140 [==============================] - 29s 210ms/step - loss: 6.7507e-05 - mean_absolute_error: 0.0056 - val_loss: 1.9324e-04 - val_mean_absolute_error: 0.0116
Epoch 6/30
140/140 [==============================] - 25s 175ms/step - loss: 4.3435e-05 - mean_absolute_error: 0.0045 - val_loss: 4.0348e-04 - val_mean_absolute_error: 0.0166
Epoch 7/30
140/140 [==============================] - 29s 210ms/step - loss: 4.5389e-05 - mean_absolute_error: 0.0047 - val_loss: 1.7128e-04 - val_mean_absolute_error: 0.0106
Epoch 8/30
140/140 [==============================] - 31s 222ms/step - loss: 4.0170e-05 - mean_absolute_error: 0.0042 - val_loss: 1.4447e-04 - val_mean_absolute_error: 0.0098
Epoch 9/30
140/140 [==============================] - 46s 327ms/step - loss: 4.5567e-05 - mean_absolute_error: 0.0046 - val_loss: 1.2182e-04 - val_mean_absolute_error: 0.0093
Epoch 10/30
140/140 [==============================] - 24s 172ms/step - loss: 2.8392e-05 - mean_absolute_error: 0.0036 - val_loss: 1.3206e-04 - val_mean_absolute_error: 0.0091
Epoch 11/30
140/140 [==============================] - 29s 204ms/step - loss: 2.8354e-05 - mean_absolute_error: 0.0035 - val_loss: 1.1863e-04 - val_mean_absolute_error: 0.0091
Epoch 12/30
140/140 [==============================] - 24s 172ms/step - loss: 3.0984e-05 - mean_absolute_error: 0.0037 - val_loss: 1.3196e-04 - val_mean_absolute_error: 0.0096
Epoch 13/30
140/140 [==============================] - 24s 172ms/step - loss: 4.6311e-05 - mean_absolute_error: 0.0046 - val_loss: 1.4698e-04 - val_mean_absolute_error: 0.0101
Epoch 14/30
140/140 [==============================] - 29s 205ms/step - loss: 3.4324e-05 - mean_absolute_error: 0.0038 - val_loss: 9.9349e-05 - val_mean_absolute_error: 0.0084
Epoch 15/30
140/140 [==============================] - 24s 172ms/step - loss: 5.1011e-05 - mean_absolute_error: 0.0049 - val_loss: 1.1553e-04 - val_mean_absolute_error: 0.0087
Epoch 16/30
140/140 [==============================] - 29s 206ms/step - loss: 3.0801e-05 - mean_absolute_error: 0.0035 - val_loss: 9.8269e-05 - val_mean_absolute_error: 0.0083
Epoch 17/30
140/140 [==============================] - 29s 207ms/step - loss: 2.2550e-05 - mean_absolute_error: 0.0033 - val_loss: 9.5089e-05 - val_mean_absolute_error: 0.0082
Epoch 18/30
140/140 [==============================] - 25s 174ms/step - loss: 3.8440e-05 - mean_absolute_error: 0.0040 - val_loss: 1.0677e-04 - val_mean_absolute_error: 0.0086
Epoch 19/30
140/140 [==============================] - 24s 174ms/step - loss: 2.4965e-05 - mean_absolute_error: 0.0034 - val_loss: 1.4991e-04 - val_mean_absolute_error: 0.0097
Epoch 20/30
140/140 [==============================] - 30s 211ms/step - loss: 2.1753e-05 - mean_absolute_error: 0.0032 - val_loss: 8.3716e-05 - val_mean_absolute_error: 0.0077
Epoch 21/30
140/140 [==============================] - 29s 209ms/step - loss: 2.2372e-05 - mean_absolute_error: 0.0029 - val_loss: 8.0377e-05 - val_mean_absolute_error: 0.0076
Epoch 22/30
140/140 [==============================] - 25s 176ms/step - loss: 3.0919e-05 - mean_absolute_error: 0.0041 - val_loss: 1.1954e-04 - val_mean_absolute_error: 0.0090
Epoch 23/30
140/140 [==============================] - 25s 178ms/step - loss: 2.8426e-05 - mean_absolute_error: 0.0038 - val_loss: 1.7505e-04 - val_mean_absolute_error: 0.0108
Epoch 24/30
140/140 [==============================] - 25s 176ms/step - loss: 2.6319e-05 - mean_absolute_error: 0.0034 - val_loss: 9.6426e-05 - val_mean_absolute_error: 0.0079
Epoch 25/30
140/140 [==============================] - 25s 176ms/step - loss: 2.5734e-05 - mean_absolute_error: 0.0035 - val_loss: 1.2537e-04 - val_mean_absolute_error: 0.0091
Epoch 26/30
140/140 [==============================] - 24s 173ms/step - loss: 2.0110e-05 - mean_absolute_error: 0.0031 - val_loss: 1.0906e-04 - val_mean_absolute_error: 0.0083
Epoch 27/30
140/140 [==============================] - 29s 208ms/step - loss: 2.1979e-05 - mean_absolute_error: 0.0032 - val_loss: 7.5809e-05 - val_mean_absolute_error: 0.0071
Epoch 28/30
140/140 [==============================] - 29s 207ms/step - loss: 2.4233e-05 - mean_absolute_error: 0.0033 - val_loss: 6.6209e-05 - val_mean_absolute_error: 0.0070
Epoch 29/30
140/140 [==============================] - 24s 173ms/step - loss: 1.8824e-05 - mean_absolute_error: 0.0029 - val_loss: 2.7348e-04 - val_mean_absolute_error: 0.0143
Epoch 30/30
140/140 [==============================] - 24s 172ms/step - loss: 2.1314e-05 - mean_absolute_error: 0.0031 - val_loss: 7.8678e-05 - val_mean_absolute_error: 0.0075

# Let's plot the mean_squared_error and mean_absolute_error using model history on the entire dataset
plot_loss_and_mae_validation(
    model_name="model_128_64_64_8_8_1_30_6_months_1_layer_dropout",
    model_history=history_128_64_64_8_8_1_30_6_months_1_layer_dropout,
)

Observations

Using six months of data as a window to predict stock prices and adding one batch normalization layer to the input training samples, the training loss and mean absolute error are decreasing gradually. In contrast, the validation loss and mean absolute error are fluctuating, i.e., both increasing and decreasing. The observed validation mean absolute error is greater than that obtained using a high-performance model with 3 LSTM layers and 3 Dense layers. It is significantly less than when using noise levels of 0.005, 0.03, and 0.01 with the same six-month data window and less than when using scaling constants of 0.1, and 0.3. One option to potentially decrease the validation mean absolute error further is to adjust the scaling magnitude.

• 3 LSTM(128-64-64) 3 Dense(8-8-1) - 6 months - 1 Dropout layer
• Epochs: 30
• Batch Size: 16
• Training Loss: 2.1314e-05
• Validation Loss: 2.7959e-04
• Training MAE: 0.0031
• Validation MAE: 0.0075

Model Building with 3 LSTM and 3 Dense layers (128-64-64-8-8-1) and 30 epochs - 6 months - Dropout (2 Layers)

# Let's create a keras sequential model
model_128_64_64_8_8_1_30_6_months_2_layer_dropout = Sequential()

# Let's add two LSTM layer's to the model network
model_128_64_64_8_8_1_30_6_months_2_layer_dropout.add(
    LSTM(128, return_sequences=True, input_shape=(x_train_6_months.shape[1], 1))
)
# Let's add Dropout Layer
model_128_64_64_8_8_1_30_6_months_2_layer_dropout.add(Dropout(0.2))
model_128_64_64_8_8_1_30_6_months_2_layer_dropout.add(LSTM(64, return_sequences=True))
model_128_64_64_8_8_1_30_6_months_2_layer_dropout.add(LSTM(64, return_sequences=False))
model_128_64_64_8_8_1_30_6_months_2_layer_dropout.add(Dropout(0.2))

# Let's add dense layer to the model network
model_128_64_64_8_8_1_30_6_months_2_layer_dropout.add(Dense(8))
model_128_64_64_8_8_1_30_6_months_2_layer_dropout.add(Dense(8))
model_128_64_64_8_8_1_30_6_months_2_layer_dropout.add(Dense(1))

# Let's build the model using mean_squared_error as the loss function
# and mean_absolute_error as the evaluation metric during the compilation process
model_128_64_64_8_8_1_30_6_months_2_layer_dropout.compile(
    optimizer="adam", loss="mean_squared_error", metrics=["mean_absolute_error"]
)

# Let's define the callbacks for Model saving and Early stopping
cb_check = keras.callbacks.ModelCheckpoint(
    filepath="model_128_64_64_8_8_1_30_6_months_2_layer_dropout_checkpoint_filepath",
    save_best_only=True,
    monitor="val_loss",
)

cb_early = keras.callbacks.EarlyStopping(monitor="val_loss", patience=30)

# Let's fit the sequential model with input features and output label
history_128_64_64_8_8_1_30_6_months_2_layer_dropout = (
    model_128_64_64_8_8_1_30_6_months_2_layer_dropout.fit(
        x_train_6_months,
        y_train_6_months,
        batch_size=16,
        epochs=30,
        callbacks=[cb_check, cb_early],
        validation_data=(x_valid_6_months, y_valid_6_months),
    )
)

Epoch 1/30
140/140 [==============================] - 48s 316ms/step - loss: 2.2793e-04 - mean_absolute_error: 0.0090 - val_loss: 4.0502e-04 - val_mean_absolute_error: 0.0162
Epoch 2/30
140/140 [==============================] - 29s 206ms/step - loss: 1.1016e-04 - mean_absolute_error: 0.0069 - val_loss: 3.2464e-04 - val_mean_absolute_error: 0.0151
Epoch 3/30
140/140 [==============================] - 29s 210ms/step - loss: 7.9938e-05 - mean_absolute_error: 0.0058 - val_loss: 2.9277e-04 - val_mean_absolute_error: 0.0144
Epoch 4/30
140/140 [==============================] - 24s 172ms/step - loss: 8.8213e-05 - mean_absolute_error: 0.0061 - val_loss: 3.3850e-04 - val_mean_absolute_error: 0.0141
Epoch 5/30
140/140 [==============================] - 24s 173ms/step - loss: 1.0610e-04 - mean_absolute_error: 0.0068 - val_loss: 3.3435e-04 - val_mean_absolute_error: 0.0142
Epoch 6/30
140/140 [==============================] - 24s 172ms/step - loss: 1.0584e-04 - mean_absolute_error: 0.0067 - val_loss: 3.5156e-04 - val_mean_absolute_error: 0.0147
Epoch 7/30
140/140 [==============================] - 30s 212ms/step - loss: 6.6620e-05 - mean_absolute_error: 0.0053 - val_loss: 2.4869e-04 - val_mean_absolute_error: 0.0122
Epoch 8/30
140/140 [==============================] - 25s 175ms/step - loss: 6.9193e-05 - mean_absolute_error: 0.0053 - val_loss: 5.5636e-04 - val_mean_absolute_error: 0.0200
Epoch 9/30
140/140 [==============================] - 25s 179ms/step - loss: 7.3110e-05 - mean_absolute_error: 0.0059 - val_loss: 2.5274e-04 - val_mean_absolute_error: 0.0124
Epoch 10/30
140/140 [==============================] - 24s 173ms/step - loss: 1.3141e-04 - mean_absolute_error: 0.0074 - val_loss: 6.0599e-04 - val_mean_absolute_error: 0.0208
Epoch 11/30
140/140 [==============================] - 29s 210ms/step - loss: 8.4906e-05 - mean_absolute_error: 0.0061 - val_loss: 1.4508e-04 - val_mean_absolute_error: 0.0097
Epoch 12/30
140/140 [==============================] - 29s 209ms/step - loss: 5.6629e-05 - mean_absolute_error: 0.0049 - val_loss: 1.2304e-04 - val_mean_absolute_error: 0.0091
Epoch 13/30
140/140 [==============================] - 25s 174ms/step - loss: 5.9792e-05 - mean_absolute_error: 0.0049 - val_loss: 1.2780e-04 - val_mean_absolute_error: 0.0092
Epoch 14/30
140/140 [==============================] - 24s 173ms/step - loss: 5.9526e-05 - mean_absolute_error: 0.0050 - val_loss: 1.4095e-04 - val_mean_absolute_error: 0.0097
Epoch 15/30
140/140 [==============================] - 24s 173ms/step - loss: 6.2485e-05 - mean_absolute_error: 0.0054 - val_loss: 1.2625e-04 - val_mean_absolute_error: 0.0092
Epoch 16/30
140/140 [==============================] - 29s 209ms/step - loss: 5.0316e-05 - mean_absolute_error: 0.0046 - val_loss: 1.1342e-04 - val_mean_absolute_error: 0.0088
Epoch 17/30
140/140 [==============================] - 25s 176ms/step - loss: 5.4554e-05 - mean_absolute_error: 0.0050 - val_loss: 1.8880e-04 - val_mean_absolute_error: 0.0114
Epoch 18/30
140/140 [==============================] - 24s 172ms/step - loss: 6.0816e-05 - mean_absolute_error: 0.0055 - val_loss: 1.2301e-04 - val_mean_absolute_error: 0.0091
Epoch 19/30
140/140 [==============================] - 31s 219ms/step - loss: 5.8447e-05 - mean_absolute_error: 0.0050 - val_loss: 1.0843e-04 - val_mean_absolute_error: 0.0086
Epoch 20/30
140/140 [==============================] - 24s 174ms/step - loss: 7.9487e-05 - mean_absolute_error: 0.0056 - val_loss: 1.3711e-04 - val_mean_absolute_error: 0.0093
Epoch 21/30
140/140 [==============================] - 24s 173ms/step - loss: 5.7083e-05 - mean_absolute_error: 0.0049 - val_loss: 1.3617e-04 - val_mean_absolute_error: 0.0091
Epoch 22/30
140/140 [==============================] - 24s 174ms/step - loss: 4.0501e-05 - mean_absolute_error: 0.0044 - val_loss: 1.4623e-04 - val_mean_absolute_error: 0.0094
Epoch 23/30
140/140 [==============================] - 24s 174ms/step - loss: 5.9763e-05 - mean_absolute_error: 0.0054 - val_loss: 1.4677e-04 - val_mean_absolute_error: 0.0100
Epoch 24/30
140/140 [==============================] - 29s 209ms/step - loss: 5.2611e-05 - mean_absolute_error: 0.0050 - val_loss: 8.7800e-05 - val_mean_absolute_error: 0.0079
Epoch 25/30
140/140 [==============================] - 24s 174ms/step - loss: 5.3540e-05 - mean_absolute_error: 0.0049 - val_loss: 1.7214e-04 - val_mean_absolute_error: 0.0108
Epoch 26/30
140/140 [==============================] - 25s 175ms/step - loss: 5.5421e-05 - mean_absolute_error: 0.0049 - val_loss: 1.5318e-04 - val_mean_absolute_error: 0.0097
Epoch 27/30
140/140 [==============================] - 25s 176ms/step - loss: 8.9474e-05 - mean_absolute_error: 0.0065 - val_loss: 8.9725e-05 - val_mean_absolute_error: 0.0078
Epoch 28/30
140/140 [==============================] - 25s 176ms/step - loss: 5.7262e-05 - mean_absolute_error: 0.0051 - val_loss: 1.7823e-04 - val_mean_absolute_error: 0.0111
Epoch 29/30
140/140 [==============================] - 30s 214ms/step - loss: 5.5485e-05 - mean_absolute_error: 0.0052 - val_loss: 8.1612e-05 - val_mean_absolute_error: 0.0076
Epoch 30/30
140/140 [==============================] - 91s 656ms/step - loss: 4.8052e-05 - mean_absolute_error: 0.0045 - val_loss: 8.0147e-05 - val_mean_absolute_error: 0.0074

# Let's plot the mean_squared_error and mean_absolute_error using model history on the entire dataset
plot_loss_and_mae_validation(
    model_name="model_128_64_64_8_8_1_30_6_months_2_layer_dropout",
    model_history=history_128_64_64_8_8_1_30_6_months_2_layer_dropout,
)

Observations

Using six months of data as a window to predict stock prices and adding one batch normalization layer to the input training samples, the training loss and mean absolute error are decreasing gradually. In contrast, the validation loss and mean absolute error are fluctuating, i.e., both increasing and decreasing. The observed validation mean absolute error is greater than that obtained using a high-performance model with 3 LSTM layers and 3 Dense layers. It is significantly less than when using noise levels of 0.005, 0.03, and 0.01 with the same six-month data window and less than when using scaling constants of 0.1, and 0.3. One option to potentially decrease the validation mean absolute error further is to adjust the scaling magnitude.

• 3 LSTM(128-64-64) 3 Dense(8-8-1) - 6 months - 1 Dropout layer
• Epochs: 30
• Batch Size: 16
• Training Loss: 4.8052e-05
• Validation Loss: 8.0147e-05
• Training MAE: 0.0045
• Validation MAE: 0.0074

# Let's define the list of models that are used for validation and comparison.
model_validation_regularization = {
    "model_128_64_64_8_8_1_30_6_months_1_layer_batch_normalization": model_128_64_64_8_8_1_30_6_months_1_layer_batch_normalization,
    "model_128_64_64_8_8_1_30_6_months_2_layer_batch_normalization": model_128_64_64_8_8_1_30_6_months_2_layer_batch_normalization,
    "model_128_64_64_8_8_1_30_6_months_1_layer_dropout": model_128_64_64_8_8_1_30_6_months_1_layer_dropout,
    "model_128_64_64_8_8_1_30_6_months_2_layer_dropout": model_128_64_64_8_8_1_30_6_months_2_layer_dropout,
}

# Let's define the list of models that are used for validation and comparison.
model_validation_regularization_plot = {
    "model_128_64_64_8_8_1_30_6_months_1_layer_dropout": model_128_64_64_8_8_1_30_6_months_1_layer_dropout,
}

validation_testing_6_months_regularization = []

for model_name in model_validation_regularization.keys():
    if "6_months" in model_name:
        model = keras.models.load_model(f"{model_name}_checkpoint_filepath")
        six_months_metrics_dict_regularization = get_metrics(period=6, model=model)
        plot_metrics_and_predictions(
            model_name=model_name,
            actual_prices=six_months_metrics_dict_regularization.get(
                "valid_actual_prices"
            ),
            predicted_prices=six_months_metrics_dict_regularization.get(
                "valid_predicted_prices"
            ),
        )
        validation_testing_6_months_regularization.append(
            {
                "Model Configuration": model_name,
                "period": "6 months",
                "Validation MAE": six_months_metrics_dict_regularization.get(
                    "validation_mae"
                ),
                "Validation MAPE": six_months_metrics_dict_regularization.get(
                    "validation_mape"
                ),
                "Test MAE": six_months_metrics_dict_regularization.get("test_mae"),
                "Test MAPE": six_months_metrics_dict_regularization.get("test_mape"),
            }
        )

Printing Validation MSE and MAE for all the regularization models - 6 months

# Let's convert the validation_testing list of dictionaries into a pandas data_frame
validation_testing_6_months_regularization_df = pd.DataFrame(
    validation_testing_6_months_regularization
)
validation_testing_6_months_regularization_df

	Model Configuration	period	Validation MAE	Validation MAPE	Test MAE	Test MAPE
0	model_128_64_64_8_8_1_30_6_months_1_layer_batc...	6 months	6.799067	0.044139	247.337539	0.331576
1	model_128_64_64_8_8_1_30_6_months_2_layer_batc...	6 months	8.567218	0.055651	168.914832	0.233823
2	model_128_64_64_8_8_1_30_6_months_1_layer_dropout	6 months	6.603170	0.042127	21.736594	0.030256
3	model_128_64_64_8_8_1_30_6_months_2_layer_dropout	6 months	7.053871	0.045771	19.266117	0.027392

Observations

---

📊The table above, which shows comparisons of different LSTM model configurations with either one or two batch normalization or dropout layers, represents a distinct configuration in each row. These models were trained using a 6-month window with three LSTM and three Dense layers to predict daily stock prices. From the table, it is evident that the LSTM model trained with one dropout layer at a rate of 0.2 and a 6-month window outperformed those trained with one or two batch normalization layers and two dropout layers. 📈

💡The best model among the different configurations of LSTM layers and neurons is the one with three LSTM layers (128, 64, and 64 neurons, respectively) and three dense layers (8, 8, and 1 neurons, respectively) with one dropout layer🏆

Powerful Architecture

from prophet import Prophet
from datetime import timedelta
from keras.layers import Bidirectional

/Users/avinash-chaluvadi/miniforge3/envs/deep-learning/lib/python3.8/site-packages/tqdm/auto.py:21: TqdmWarning:

IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html

Bidirectional LSTM

def create_bidirectional_lstm_model(input_shape):
    model = Sequential(
        [
            # First layer needs to specify input_shape and should return sequences
            Bidirectional(LSTM(128, return_sequences=True), input_shape=input_shape),
            Dropout(0.2),
            # Intermediate LSTM layers should also return sequences except for the last LSTM layer
            Bidirectional(LSTM(64, return_sequences=True)),
            Dropout(0.2),
            Bidirectional(
                LSTM(32, return_sequences=True)
            ),  # If you want more layers, they should also return sequences
            Dropout(0.2),
            # Only the last LSTM layer should not return sequences unless followed by another LSTM layer
            Bidirectional(LSTM(64, return_sequences=True)),
            Dropout(0.2),
            Bidirectional(LSTM(64, return_sequences=True)),
            Dropout(0.2),
            Bidirectional(LSTM(64)),
            Dropout(0.2),
            Dense(25),
            Dropout(0.2),
            Dense(1),
        ]
    )
    return model


# Assume your input shape from the data preprocessing step is (180, 2) for each sample
model_powerful_architecture_bidirectional = create_bidirectional_lstm_model(
    input_shape=(x_train_6_months.shape[1], 1)
)
model_powerful_architecture_bidirectional.summary()

Model: "sequential_72"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 bidirectional_20 (Bidirect  (None, 180, 256)          133120    
 ional)                                                          
                                                                 
 dropout_21 (Dropout)        (None, 180, 256)          0         
                                                                 
 bidirectional_21 (Bidirect  (None, 180, 128)          164352    
 ional)                                                          
                                                                 
 dropout_22 (Dropout)        (None, 180, 128)          0         
                                                                 
 bidirectional_22 (Bidirect  (None, 180, 64)           41216     
 ional)                                                          
                                                                 
 dropout_23 (Dropout)        (None, 180, 64)           0         
                                                                 
 bidirectional_23 (Bidirect  (None, 180, 128)          66048     
 ional)                                                          
                                                                 
 dropout_24 (Dropout)        (None, 180, 128)          0         
                                                                 
 bidirectional_24 (Bidirect  (None, 180, 128)          98816     
 ional)                                                          
                                                                 
 dropout_25 (Dropout)        (None, 180, 128)          0         
                                                                 
 bidirectional_25 (Bidirect  (None, 128)               98816     
 ional)                                                          
                                                                 
 dropout_26 (Dropout)        (None, 128)               0         
                                                                 
 dense_177 (Dense)           (None, 25)                3225      
                                                                 
 dropout_27 (Dropout)        (None, 25)                0         
                                                                 
 dense_178 (Dense)           (None, 1)                 26        
                                                                 
=================================================================
Total params: 605619 (2.31 MB)
Trainable params: 605619 (2.31 MB)
Non-trainable params: 0 (0.00 Byte)
_________________________________________________________________

# Let's build the model using mean_squared_error as the loss function
# and mean_absolute_error as the evaluation metric during the compilation process
model_powerful_architecture_bidirectional.compile(
    optimizer="adam", loss="mean_squared_error", metrics=["mean_absolute_error"]
)

# Let's define the callbacks for Model saving and Early stopping
cb_check = keras.callbacks.ModelCheckpoint(
    filepath="model_powerful_architecture_bidirectional_checkpoint_filepath",
    save_best_only=True,
    monitor="val_loss",
)

cb_early = keras.callbacks.EarlyStopping(monitor="val_loss", patience=30)

# Let's fit the sequential model with input features and output label
history_powerful_architecture_bidirectional = (
    model_powerful_architecture_bidirectional.fit(
        x_train_6_months,
        y_train_6_months,
        batch_size=16,
        epochs=30,
        callbacks=[cb_check, cb_early],
        validation_data=(x_valid_6_months, y_valid_6_months),
    )
)

Epoch 1/30
140/140 [==============================] - 123s 764ms/step - loss: 7.6105e-04 - mean_absolute_error: 0.0172 - val_loss: 8.6744e-04 - val_mean_absolute_error: 0.0237
Epoch 2/30
140/140 [==============================] - 92s 658ms/step - loss: 2.8311e-04 - mean_absolute_error: 0.0107 - val_loss: 7.5064e-04 - val_mean_absolute_error: 0.0223
Epoch 3/30
140/140 [==============================] - 112s 796ms/step - loss: 2.9500e-04 - mean_absolute_error: 0.0107 - val_loss: 6.9599e-04 - val_mean_absolute_error: 0.0212
Epoch 4/30
140/140 [==============================] - 78s 554ms/step - loss: 3.0989e-04 - mean_absolute_error: 0.0115 - val_loss: 0.0024 - val_mean_absolute_error: 0.0436
Epoch 5/30
140/140 [==============================] - 151s 1s/step - loss: 2.4741e-04 - mean_absolute_error: 0.0105 - val_loss: 4.0965e-04 - val_mean_absolute_error: 0.0167
Epoch 6/30
140/140 [==============================] - 57s 407ms/step - loss: 2.2896e-04 - mean_absolute_error: 0.0098 - val_loss: 9.9035e-04 - val_mean_absolute_error: 0.0264
Epoch 7/30
140/140 [==============================] - 57s 404ms/step - loss: 1.9862e-04 - mean_absolute_error: 0.0095 - val_loss: 4.5378e-04 - val_mean_absolute_error: 0.0171
Epoch 8/30
140/140 [==============================] - 109s 783ms/step - loss: 1.5452e-04 - mean_absolute_error: 0.0085 - val_loss: 3.0609e-04 - val_mean_absolute_error: 0.0145
Epoch 9/30
140/140 [==============================] - 124s 889ms/step - loss: 1.4090e-04 - mean_absolute_error: 0.0080 - val_loss: 2.3329e-04 - val_mean_absolute_error: 0.0125
Epoch 10/30
140/140 [==============================] - 63s 447ms/step - loss: 2.5891e-04 - mean_absolute_error: 0.0108 - val_loss: 8.4436e-04 - val_mean_absolute_error: 0.0232
Epoch 11/30
140/140 [==============================] - 64s 455ms/step - loss: 2.1733e-04 - mean_absolute_error: 0.0099 - val_loss: 2.8298e-04 - val_mean_absolute_error: 0.0132
Epoch 12/30
140/140 [==============================] - 65s 459ms/step - loss: 1.6440e-04 - mean_absolute_error: 0.0085 - val_loss: 2.9157e-04 - val_mean_absolute_error: 0.0135
Epoch 13/30
140/140 [==============================] - 57s 405ms/step - loss: 2.6002e-04 - mean_absolute_error: 0.0105 - val_loss: 3.1000e-04 - val_mean_absolute_error: 0.0145
Epoch 14/30
140/140 [==============================] - 66s 474ms/step - loss: 1.6089e-04 - mean_absolute_error: 0.0084 - val_loss: 2.4595e-04 - val_mean_absolute_error: 0.0129
Epoch 15/30
140/140 [==============================] - 64s 452ms/step - loss: 1.7662e-04 - mean_absolute_error: 0.0088 - val_loss: 3.6694e-04 - val_mean_absolute_error: 0.0159
Epoch 16/30
140/140 [==============================] - 98s 700ms/step - loss: 2.0641e-04 - mean_absolute_error: 0.0095 - val_loss: 2.2360e-04 - val_mean_absolute_error: 0.0122
Epoch 17/30
140/140 [==============================] - 82s 581ms/step - loss: 1.4572e-04 - mean_absolute_error: 0.0079 - val_loss: 3.5114e-04 - val_mean_absolute_error: 0.0155
Epoch 18/30
140/140 [==============================] - 63s 451ms/step - loss: 1.4617e-04 - mean_absolute_error: 0.0080 - val_loss: 7.8470e-04 - val_mean_absolute_error: 0.0235
Epoch 19/30
140/140 [==============================] - 131s 939ms/step - loss: 1.6714e-04 - mean_absolute_error: 0.0086 - val_loss: 1.6267e-04 - val_mean_absolute_error: 0.0102
Epoch 20/30
140/140 [==============================] - 58s 407ms/step - loss: 1.5577e-04 - mean_absolute_error: 0.0081 - val_loss: 2.4212e-04 - val_mean_absolute_error: 0.0123
Epoch 21/30
140/140 [==============================] - 55s 395ms/step - loss: 1.9935e-04 - mean_absolute_error: 0.0092 - val_loss: 6.6522e-04 - val_mean_absolute_error: 0.0213
Epoch 22/30
140/140 [==============================] - 54s 388ms/step - loss: 1.7202e-04 - mean_absolute_error: 0.0087 - val_loss: 1.7305e-04 - val_mean_absolute_error: 0.0108
Epoch 23/30
140/140 [==============================] - 60s 428ms/step - loss: 1.7822e-04 - mean_absolute_error: 0.0085 - val_loss: 2.3747e-04 - val_mean_absolute_error: 0.0128
Epoch 24/30
140/140 [==============================] - 59s 422ms/step - loss: 1.3577e-04 - mean_absolute_error: 0.0077 - val_loss: 9.1783e-04 - val_mean_absolute_error: 0.0278
Epoch 25/30
140/140 [==============================] - 57s 409ms/step - loss: 1.2430e-04 - mean_absolute_error: 0.0075 - val_loss: 3.1069e-04 - val_mean_absolute_error: 0.0141
Epoch 26/30
140/140 [==============================] - 57s 404ms/step - loss: 1.3246e-04 - mean_absolute_error: 0.0077 - val_loss: 2.2032e-04 - val_mean_absolute_error: 0.0122
Epoch 27/30
140/140 [==============================] - 58s 417ms/step - loss: 1.3816e-04 - mean_absolute_error: 0.0077 - val_loss: 2.3835e-04 - val_mean_absolute_error: 0.0128
Epoch 28/30
140/140 [==============================] - 59s 419ms/step - loss: 1.6075e-04 - mean_absolute_error: 0.0082 - val_loss: 1.9812e-04 - val_mean_absolute_error: 0.0116
Epoch 29/30
140/140 [==============================] - 64s 458ms/step - loss: 1.3580e-04 - mean_absolute_error: 0.0075 - val_loss: 4.1407e-04 - val_mean_absolute_error: 0.0177
Epoch 30/30
140/140 [==============================] - 60s 430ms/step - loss: 1.6067e-04 - mean_absolute_error: 0.0083 - val_loss: 1.6935e-04 - val_mean_absolute_error: 0.0104

# Let's plot the mean_squared_error and mean_absolute_error using model history on the entire dataset
plot_loss_and_mae_validation(
    model_name="model_powerful_architecture_bidirectional",
    model_history=history_powerful_architecture_bidirectional,
)

# Let's define the list of models that are used for validation and comparison.
model_validation_powerful_architecture = {
    "model_powerful_architecture_bidirectional": model_powerful_architecture_bidirectional,
}

validation_testing_6_months_powerful_architecture = []

for model_name in model_validation_powerful_architecture.keys():
    model = keras.models.load_model(f"{model_name}_checkpoint_filepath")
    six_months_metrics_dict_powerful_architecture = get_metrics(period=6, model=model)
    plot_metrics_and_predictions(
        model_name=model_name,
        actual_prices=six_months_metrics_dict_powerful_architecture.get(
            "valid_actual_prices"
        ),
        predicted_prices=six_months_metrics_dict_powerful_architecture.get(
            "valid_predicted_prices"
        ),
    )
    validation_testing_6_months_powerful_architecture.append(
        {
            "Model Configuration": model_name,
            "period": "6 months",
            "Validation MAE": six_months_metrics_dict_powerful_architecture.get(
                "validation_mae"
            ),
            "Validation MAPE": six_months_metrics_dict_powerful_architecture.get(
                "validation_mape"
            ),
            "Test MAE": six_months_metrics_dict_powerful_architecture.get("test_mae"),
            "Test MAPE": six_months_metrics_dict_powerful_architecture.get("test_mape"),
        }
    )

Printing Validation MSE and MAE for all the regularization models - 6 months

# Let's convert the validation_testing list of dictionaries into a pandas data_frame
validation_testing_6_months_powerful_architecture_df = pd.DataFrame(
    validation_testing_6_months_powerful_architecture
)
validation_testing_6_months_powerful_architecture_df

	Model Configuration	period	Validation MAE	Validation MAPE	Test MAE	Test MAPE
0	model_powerful_architecture_bidirectional	6 months	9.702311	0.064795	33.59239	0.046592

Observations

Using a powerful architecture like stacked bidirectional LSTM, the training loss and training MAE are gradually decreasing; however, the validation loss and validation MAE are fluctuating, i.e., increasing and decreasing. The observed validation MAE is significantly higher than that of the base model with three LSTM layers (128-64-64) and three Dense layers (8-8-1). Additionally, the observed MAE using bidirectional LSTM is significantly higher than that achieved by applying augmentation techniques or using dropout and batch normalization.

• Epochs: 30
• Batch Size: 16
• Training Loss: 1.6067e-04
• Validation Loss: 1.6935e-04
• Training MAE: 0.0083
• Validation MAE: 0.0104

Pre-Trained Model - Prophet

# Let's reset_index of nvidia_data_frame to prepare the data_frame for prophet
df_prophet = nvidia_data_frame_viz.reset_index().rename(
    columns={"Date": "ds", "Close": "y"}
)

# Split the data into train and test sets
train_size = int(0.9 * len(df_prophet))
train_df = df_prophet.iloc[:train_size]
test_df = df_prophet.iloc[train_size:]

# Initialize and fit the Prophet model
model = Prophet(daily_seasonality=True)
model.fit(train_df)

# Predict on the test set
future = model.make_future_dataframe(periods=len(test_df), freq="D")
forecast = model.predict(future)

# Extract predictions and compare them with the actual test values
preds = forecast["yhat"][train_size:]

DEBUG:cmdstanpy:input tempfile: /var/folders/v5/h98tv59n1nb13jdykh4_dy2r0000gn/T/tmp0a2dv4tf/04ojwahe.json
DEBUG:cmdstanpy:input tempfile: /var/folders/v5/h98tv59n1nb13jdykh4_dy2r0000gn/T/tmp0a2dv4tf/6by5yv0w.json
DEBUG:cmdstanpy:idx 0
DEBUG:cmdstanpy:running CmdStan, num_threads: None
DEBUG:cmdstanpy:CmdStan args: ['/Users/avinash-chaluvadi/miniforge3/envs/deep-learning/lib/python3.8/site-packages/prophet/stan_model/prophet_model.bin', 'random', 'seed=90758', 'data', 'file=/var/folders/v5/h98tv59n1nb13jdykh4_dy2r0000gn/T/tmp0a2dv4tf/04ojwahe.json', 'init=/var/folders/v5/h98tv59n1nb13jdykh4_dy2r0000gn/T/tmp0a2dv4tf/6by5yv0w.json', 'output', 'file=/var/folders/v5/h98tv59n1nb13jdykh4_dy2r0000gn/T/tmp0a2dv4tf/prophet_modelqvbg8f8u/prophet_model-20240506232347.csv', 'method=optimize', 'algorithm=lbfgs', 'iter=10000']
23:23:47 - cmdstanpy - INFO - Chain [1] start processing
INFO:cmdstanpy:Chain [1] start processing
python(87993) MallocStackLogging: can't turn off malloc stack logging because it was not enabled.
23:23:48 - cmdstanpy - INFO - Chain [1] done processing
INFO:cmdstanpy:Chain [1] done processing

# Plotting training results
fig, ax = plt.subplots(figsize=(9, 4), dpi=300)
ax.plot(train_df["ds"], train_df["y"], label="Actual Train", color="blue")
ax.plot(
    future["ds"][:train_size],
    forecast["yhat"][:train_size],
    label="Predicted Train",
    color="red",
    linestyle="--",
)
ax.set_title("Training Data: Actual vs. Predicted")
ax.set_xlabel("Date")
ax.set_ylabel("Stock Price")
ax.legend()

# Plotting testing results
fig, ax = plt.subplots(figsize=(9, 4), dpi=300)
ax.plot(test_df["ds"], test_df["y"], label="Actual Test", color="green")
ax.plot(
    future["ds"][train_size:],
    preds,
    label="Predicted Test",
    color="purple",
    linestyle="--",
)
ax.set_title("Testing Data: Actual vs. Predicted")
ax.set_xlabel("Date")
ax.set_ylabel("Stock Price")
ax.legend()

plt.show()

# Calculate Mean Squared Error and Mean Absolute Error
mse = mean_squared_error(test_df["y"], preds)
mae = mean_absolute_error(test_df["y"], preds)
mape = mean_absolute_percentage_error(test_df["y"], preds)

print(f"Mean Squared Error             -->{mse}")
print(f"Mean Absolute Error            -->{mae}")
print(f"Mean Absolute Percentage Error -->{mape}")

Mean Squared Error             -->117081.32802437257
Mean Absolute Error            -->284.2993087597024
Mean Absolute Percentage Error -->0.5114114016420485

Observations

Using a pre-trained model like Facebook Prophet. The observed normalized validation MAE is significantly higher than that of the base model with three LSTM layers (128-64-64) and three Dense layers (8-8-1). Additionally, the observed MAE using prophet is significantly higher than that achieved by applying augmentation techniques or using dropout and batch normalization.

Predicting Nvidia's stock prices for the next year

model = Prophet(
    daily_seasonality=True
)  # Choose seasonality based on your data's frequency
model.fit(df_prophet)

# Create future dates of 365 days
future = model.make_future_dataframe(periods=365)

# Predict the future
forecast = model.predict(future)

DEBUG:cmdstanpy:input tempfile: /var/folders/v5/h98tv59n1nb13jdykh4_dy2r0000gn/T/tmp0a2dv4tf/27glq7_x.json
DEBUG:cmdstanpy:input tempfile: /var/folders/v5/h98tv59n1nb13jdykh4_dy2r0000gn/T/tmp0a2dv4tf/qy2f8y1x.json
DEBUG:cmdstanpy:idx 0
DEBUG:cmdstanpy:running CmdStan, num_threads: None
DEBUG:cmdstanpy:CmdStan args: ['/Users/avinash-chaluvadi/miniforge3/envs/deep-learning/lib/python3.8/site-packages/prophet/stan_model/prophet_model.bin', 'random', 'seed=16562', 'data', 'file=/var/folders/v5/h98tv59n1nb13jdykh4_dy2r0000gn/T/tmp0a2dv4tf/27glq7_x.json', 'init=/var/folders/v5/h98tv59n1nb13jdykh4_dy2r0000gn/T/tmp0a2dv4tf/qy2f8y1x.json', 'output', 'file=/var/folders/v5/h98tv59n1nb13jdykh4_dy2r0000gn/T/tmp0a2dv4tf/prophet_model3lffre14/prophet_model-20240506105222.csv', 'method=optimize', 'algorithm=lbfgs', 'iter=10000']
10:52:22 - cmdstanpy - INFO - Chain [1] start processing
INFO:cmdstanpy:Chain [1] start processing
python(79118) MallocStackLogging: can't turn off malloc stack logging because it was not enabled.
10:52:23 - cmdstanpy - INFO - Chain [1] done processing
INFO:cmdstanpy:Chain [1] done processing

# Display the forecasted values
print(forecast[["ds", "yhat", "yhat_lower", "yhat_upper"]].tail())

# Plot the forecast
fig, ax = plt.subplots(figsize=(8, 3), dpi=300)
ax.plot(
    forecast["ds"], forecast["yhat"], label="Predicted", color="blue"
)  # Predicted mean
ax.fill_between(
    forecast["ds"],
    forecast["yhat_lower"],
    forecast["yhat_upper"],
    color="gray",
    alpha=0.5,
    label="Uncertainty Interval",
)  # Uncertainty interval
ax.plot(
    df_prophet["ds"], df_prophet["y"], label="Actual", color="black", linestyle="--"
)  # Optional: plot actual values if you have them

# Setting labels and titles
ax.set_title("Nvidia Stock Price Forecast")
ax.set_xlabel("Date")
ax.set_ylabel("Stock Price")

# Add legend
ax.legend()

plt.show()

# Plot the components
fig_components = model.plot_components(forecast)

             ds      yhat  yhat_lower  yhat_upper
3379 2025-04-27  0.810246    0.727313    0.894119
3380 2025-04-28  0.810117    0.724349    0.898540
3381 2025-04-29  0.808490    0.721147    0.893309
3382 2025-04-30  0.807260    0.723628    0.890476
3383 2025-05-01  0.806523    0.722231    0.892449

| Model | Architecture | Test MAE | Test MAPE | |------------------------|----------------------------------------------|-----------------------|----------------------| | Powerful Architecture | 6 Bidirectional LSTM, 2 Dense layers | 0.0104 | 0.046592 | | Pretrained Model (Prophet) | None | 0.3003696700815516 | 0.5148255211413744 |

Anomaly Detection - Input

# Let's import anomaly detection tool kit packages
from adtk.visualization import plot
from adtk.detector import QuantileAD

# Remove the 'day_of_week' column from the nvidia_data_frame and store the result
nvidia_data_frame_anomaly = nvidia_data_frame.drop("day_of_week", axis=1)

# Initialize an anomaly detector that identifies outliers outside the 1st and 99th percentiles
quantile_detector = QuantileAD(low=0.01, high=0.99)

# Fit the anomaly detector to the data and detect anomalies
anomalies = quantile_detector.fit_detect(nvidia_data_frame_anomaly)

# Let's plot the actual prices and identify anomalies using red marker
fig, ax = plt.subplots(figsize=(18, 8), dpi=300)
plot(
    nvidia_data_frame_anomaly,
    anomaly=anomalies,
    anomaly_color="red",
    anomaly_tag="marker",
    axes=ax,
    ts_alpha=0.9,
    anomaly_markersize=2,
    anomaly_alpha=0.3,
)

[<Axes: >]

Anomaly Detection - Output

# Predict the test data using the best performed model
best_model_predictions = model_128_64_64_8_8_1_30_6_months_validation.predict(
    x_test_6_months
)

# Calculate the Mean Absolute Error (MAE) for each prediction compared to the true values
test_mae_loss = np.mean(np.abs(best_model_predictions - y_test_6_months), axis=1)

# Create a DataFrame from the test data, skipping the first 180 days
test_score_df = pd.DataFrame(test_data_6_months[180:])
# Add the MAE loss to the DataFrame as a new column
test_score_df["loss"] = test_mae_loss
# Set a fixed threshold for anomaly detection to 0.65
test_score_df["threshold"] = 0.65
# Determine if each data point is an anomaly by comparing the loss to the threshold
test_score_df["anomaly"] = test_score_df.loss > test_score_df.threshold

# Include the closing price of the stock for each day in the DataFrame
test_score_df["close"] = test_data_6_months[180:, 0]

# Display the DataFrame
test_score_df

4/4 [==============================] - 1s 90ms/step

	0	1	loss	threshold	anomaly	close
0	0.482176	0.25	0.216444	0.65	False	0.482176
1	0.488711	0.50	0.212720	0.65	False	0.488711
2	0.492681	0.75	0.209369	0.65	False	0.492681
3	0.507303	1.00	0.206314	0.65	False	0.507303
4	0.510312	0.00	0.199778	0.65	False	0.510312
...	...	...	...	...	...	...
116	0.869401	0.75	0.188588	0.65	False	0.869401
117	0.923277	1.00	0.194594	0.65	False	0.923277
118	0.923509	0.00	0.217660	0.65	False	0.923509
119	0.909204	0.25	0.233428	0.65	False	0.909204
120	0.873719	0.50	0.225918	0.65	False	0.873719

121 rows × 6 columns

# Create a list of numbers from 0 to 120 to use as the number of days as x-axis values
x = list(range(121))
fig = go.Figure()

# Add a line plot for the test data loss values
fig.add_trace(go.Scatter(x=x, y=test_score_df.loss, mode="lines", name="Test Loss"))

# Add a line plot for the anomaly detection threshold
fig.add_trace(
    go.Scatter(x=x, y=test_score_df.threshold, mode="lines", name="Threshold")
)

# Configure the layout to display the legend and show the plot
fig.update_layout(showlegend=True)
fig.show()

The threshold in the above plot is too high to detect anomalies on test data mean absolute error effectively. Let's reduce the threshold to 0.30 and plot again to see how the model generalizes and to further discover anomalies.

# Set a fixed threshold for anomaly detection to 0.65
test_score_df["threshold"] = 0.25
# Determine if each data point is an anomaly by comparing the loss to the threshold
test_score_df["anomaly"] = test_score_df.loss > test_score_df.threshold

# Include the closing price of the stock for each day in the DataFrame
test_score_df["close"] = test_data_6_months[180:, 0]

# Display the DataFrame
test_score_df

	0	1	loss	threshold	anomaly	close
0	0.482176	0.25	0.216444	0.25	False	0.482176
1	0.488711	0.50	0.212720	0.25	False	0.488711
2	0.492681	0.75	0.209369	0.25	False	0.492681
3	0.507303	1.00	0.206314	0.25	False	0.507303
4	0.510312	0.00	0.199778	0.25	False	0.510312
...	...	...	...	...	...	...
116	0.869401	0.75	0.188588	0.25	False	0.869401
117	0.923277	1.00	0.194594	0.25	False	0.923277
118	0.923509	0.00	0.217660	0.25	False	0.923509
119	0.909204	0.25	0.233428	0.25	False	0.909204
120	0.873719	0.50	0.225918	0.25	False	0.873719

121 rows × 6 columns

# Create a list of numbers from 0 to 120 to use as the number of days as x-axis values
x = list(range(121))
fig = go.Figure()

# Add a line plot for the test data loss values
fig.add_trace(go.Scatter(x=x, y=test_score_df.loss, mode="lines", name="Test Loss"))

# Add a line plot for the anomaly detection threshold
fig.add_trace(
    go.Scatter(x=x, y=test_score_df.threshold, mode="lines", name="Threshold")
)

# Configure the layout to display the legend and show the plot
fig.update_layout(showlegend=True)
fig.show()

Conclusion

After exploring various model architectures with different configurations of LSTM and dense layers, it is clear that the highest validation accuracy was achieved using a model comprising three LSTM layers (128-64-64) and three dense layers (8-8-1). Despite experimenting with different data augmentation techniques such as jittering, scaling the magnitude of time series trend values, and applying warping to smooth the trend, no significant improvement in validation accuracy was observed. Additionally, incorporating Batch Normalization and Dropout layers did not enhance performance. Attempts to utilize more complex architectures, including a very dense Bi-directional LSTM, led to overfitting. Moreover, employing pre-trained models like Facebook Prophet did not yield better validation results. This comprehensive assessment underscores the challenges and complexities in enhancing the predictive accuracy of stock price models for Nvidia using LSTM networks.